[GE users] load balancing

Lönroth Erik erik.lonroth at scania.com
Mon Jun 4 08:20:21 BST 2007


    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Isn't this achievable simply by adding a resonable value for "mem_used" to the "Load Thresholds" in the queue configuration? It reads:
 
  "The Load Thresholds prevent the scheduling of additional jobs to the queue. A threshold can be suppled for any load value."
 
... and "mem_used" seems to be reported by SGE.
 
Lets say you have 4G of RAM. Setting "mem_used" to 2G, would prevent jobs to be dispatched onto this node if not half of the ram was still unused.
 
I'm not sure if this will have any nasty side effects tho, maybe someone can fill in on this?
 
/Erik
 
 

	-----Original Message-----
	From: John Tseng [mailto:jtseng at montalvosystems.com] 
	Sent: den 1 juni 2007 20:38
	To: users at gridengine.sunsource.net
	Subject: Re: [GE users] load balancing
	
	
	A quick and directory way:
	
	You could setup a default request of "mem_free=100M" in your sge_request file.  Then by default, all jobs will require that at least 100MB is free.
	
	You might want to add a job_load_adjustments of 100MB in the scheduler to prevent multiple jobs from consuming the same "100 MB" in the same scheduling cycle/decision loop.
	
	The better way:
	A better way to do this  would be to have a memory consumable per host and have each job "request" the amount of memory it needs.  This way all cases are accounted for.  SGE doesn't provide this by default - it only reports the "load" mem_total and doesn't automatically create a  "consumable"  mem.  (Would it be nice if sge would do this for us?)
	
	
	
	-john
	
	----- Original Message -----
	From: "Forum Parmar" <forum.parmar at gmail.com>
	To: users at gridengine.sunsource.net
	Sent: Friday, June 1, 2007 10:46:12 AM (GMT-0800) America/Los_Angeles
	Subject: [GE users] load balancing
	
	
	Hi,
	I am not sure if the engine takes into account memory for load balancing.
	Suppose there is a quad core machine and only one job is running on one core, but it is using up the entire RAM, will another job be assigned to this node just because there are free cores?
	 
	If its possible, I would like no more jobs assigned to this machine if majority of the RAM is in use.
	 
	Thank you,
	Forum




More information about the gridengine-users mailing list