[GE users] "Dispatching rate" of jobs

andre_ismll busche at ismll.de
Mon Dec 6 11:04:32 GMT 2010

    [ The following text is in the "utf-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Hi Bernhard,

Am 06.12.2010 10:21, schrieb chambon:
> Is there any solution to control the "dispathing rate" (I mean the number of jobs dispatched per scheduling pass), mainly to avoid bootleneck from jobs using storage services ?

you could possibly play around with the scheduler configuration, esp. 
with "job_load_adjustments", "load_adjustment_decay_time", and possibly 
the "load_formula" (depending on your scenario):

e.g., we made the following:
job_load_adjustments              np_load_avg=2.00
load_adjustment_decay_time        0:3:00

resulting in slowly dispatching the jobs (each job increases the load 
instantly by 2, which is then decayed within 3 mins.) This works for us, 
as we only have jobs requiring a single CPU/slot.

You could check and adjust the scheduling behaviour with an empty queue 
when submitting a bunch of jobs (more than slots are available), each 
making 100% CPU load: e.g., if using Ganglia, you could monitor a 
slow-increase of cluster load (jobs are dispatched slowly) when 
increasing load_adjustment_decay_time.

HTH (and we made nothing silly in this case),

André Busche, MSc.
Information Systems and Machine Learning Lab (ISMLL),
University of Hildesheim,
Tel: +49 (5121) 883 765


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list