[GE users] question about job queue interval
Stephan Grell - Sun Germany - SSG - Software Engineer
stephan.grell at sun.com
Thu Apr 21 19:44:06 BST 2005
[ The following text is in the "windows-1252" character set. ]
[ Your display is set for the "ISO-8859-10" character set. ]
[ Some special characters may be displayed incorrectly. ]
I added some comments.
Chris Duke wrote:
> I have a question about the job queue interval. I apologize in advance
> if the terms I use are not exactly correct, I?m fairly new to this.
> I?m using SGE 5.3p6. We use SGE for a regression test. We run about
> 2000 small jobs in a full regression test. The average test is about
> 10-20 seconds (wall clock).
> What I?ve noticed is that the queuing process on the master seems to
> run about every 15 seconds. What this occasionally causes is long gaps
> when all the queues are empty, and they don?t get populated with new
> jobs for several seconds. This delay is not a huge problem, but when
> you calculate the ratio of the delay to the execution time of the job,
> it is rather large. By reducing that delay, I am hoping to finish the
> regression test run much more quickly. I realize that there would be
> additional overhead associated with shorter intervals (because of
> increased polling), but I would like to be able to experiment and
> Solutions that seem like they would work, but I don?t know how to
> manipulate the system to do:
> * Control the poll interval of the master and all execute hosts.
> Reduce and optimize.
there is no poll interval in the grid engine. The qmaster triggers the
scheduler every scheduling interval. This can be set via qconf -msconf.
if the scheduler is busy, it gets not triggered. In the worst case, it
can lead to 100% cpu utilization by the scheduler. You might want to
take a look into the performance tuning howto on the open source side.
> * Our typical execute host has 2 CPU?s. Control the master so that
> it might put 3 jobs into a particular execute hosts queue, but
> the execute host would only be allowed to execute 2 jobs at a
> time. That way the execute host always has something to do,
> instead of periods of idle time.
you can set the number ob slots per queue. SGE does not care for the
CPUs on a machine, but for the slots. Though that allows you to controll the
number of jobs on the hosts.
> Thanks in advance for any help and ideas.
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users