[GE users] V6.1 scheduler performance

McCalla, Mac macmccalla at hess.com
Thu Apr 10 20:14:28 BST 2008

	Just thought I would post some information about V6.1 scheduler
behavior that I discovered today, in case it helps someone else.  I
recently upgraded our cluster from V6.0u7 to V6.1u3, in hopes that
scheduling performance would improve but there was no perceptible change
in  scheduling rate.  However, this morning I implemented a trivial
resource quota set to limit the max number of user jobs per host.  There
had been no previous resource quota sets defined.  Within 5 minutes, the
scheduler cpu usage on our dedicated 4-cpu qmaster/scheduler machine was
reduced from 100% of a processor (fairly typical behavior for the
current workload), to 2-20% of a processor and time from submission to
start for small jobs dropped from about 45 minutes to 15 to 30 seconds.
No other change in configuration or  workload took place (to my
knowledge).   The cluster context is fairly robust, with 2000 hosts, 12
queues, 10 parallel environments, and about 9000 queue instances.  I
would be interested to know if any others have had similar experiences.


Mac McCalla
Geoscience Systems Development Advisor
Hess Corporation
One Allen Center
500 Dallas St.
Houston, Texas  77002

More information about the gridengine-users mailing list