[GE users] load balancing

macona ashley.macon at colorado.edu
Thu Nov 4 15:26:00 GMT 2010

> Hi,
> I am not sure if the engine takes into account memory for load balancing.
> Suppose there is a quad core machine and only one job is running on one
> core, but it is using up the entire RAM, will another job be assigned to
> this node just because there are free cores?
> If its possible, I would like no more jobs assigned to this machine if
> majority of the RAM is in use.
> Thank you,
> Forum

I'll preface the following by saying our setup is new to us, but has worked well so far.

We modified the load formula to also take into account memory usage. We also set load_thresholds on queues for mem_used and swap_used and setup virtual free as a consumable. 

We have some queues which require h_vmem specification and thus will kill jobs that use more memory than requested and other queues without this requirement,
since not all of our users know their job memory needs.

Managing memory resources with SGE, IMHO, can be a tricky business, especially for non-draconian environments, as there are several subtleties to consider.


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list