[GE users] question about job queue interval

Chris Duke chris.duke at novas.com
Thu Apr 21 19:32:22 BST 2005


I have a question about the job queue interval. I apologize in advance
if the terms I use are not exactly correct, I'm fairly new to this. I'm
using SGE 5.3p6. We use SGE for a regression test. We run about 2000
small jobs in a full regression test. The average test is about 10-20
seconds (wall clock). 

 

What I've noticed is that the queuing process on the master seems to run
about every 15 seconds. What this occasionally causes is long gaps when
all the queues are empty, and they don't get populated with new jobs for
several seconds. This delay is not a huge problem, but when you
calculate the ratio of the delay to the execution time of the job, it is
rather large. By reducing that delay, I am hoping to finish the
regression test run much more quickly. I realize that there would be
additional overhead associated with shorter intervals (because of
increased polling), but I would like to be able to experiment and
optimize.

 

Solutions that seem like they would work, but I don't know how to
manipulate the system to do:

*	Control the poll interval of the master and all execute hosts.
Reduce and optimize.
*	Our typical execute host has 2 CPU's. Control the master so that
it might put 3 jobs into a particular execute hosts queue, but the
execute host would only be allowed to execute 2 jobs at a time. That way
the execute host always has something to do, instead of periods of idle
time.

 

Thanks in advance for any help and ideas.

 

--chris

 

 




More information about the gridengine-users mailing list