[GE users] RE: [GE users] load balancing
kristian at skriver.de
Thu Nov 4 13:59:06 GMT 2010
[ The following text is in the "iso-8859-1" character set. ]
[ Your display is set for the "ISO-8859-10" character set. ]
[ Some special characters may be displayed incorrectly. ]
One problem we faced using this method is, that it takes serveral minutes before a job acutally uses all the memory ie. 2 GB. So when we submit ie. 4 jobs each using 2 GB., then the Grid Master will have no problem submitting the tasks to just one compute node (having 4 cores), since the Grid Master at the time of making the decission of where to submit the task, can see that there is plenty of memory free on the compute node.
We used the consumerable resource and it works like a charm and is very flexible.
Isn't this achievable simply by adding a resonable value for "mem_used" to the "Load Thresholds" in the queue configuration? It reads:
"The Load Thresholds prevent the scheduling of additional jobs to the queue. A threshold can be suppled for any load value."
... and "mem_used" seems to be reported by SGE.
Lets say you have 4G of RAM. Setting "mem_used" to 2G, would prevent jobs to be dispatched onto this node if not half of the ram was still unused.
I'm not sure if this will have any nasty side effects tho, maybe someone can fill in on this?
More information about the gridengine-users