[GE users] V6.1 scheduler performance
macmccalla at hess.com
Thu Apr 10 20:14:28 BST 2008
Just thought I would post some information about V6.1 scheduler
behavior that I discovered today, in case it helps someone else. I
recently upgraded our cluster from V6.0u7 to V6.1u3, in hopes that
scheduling performance would improve but there was no perceptible change
in scheduling rate. However, this morning I implemented a trivial
resource quota set to limit the max number of user jobs per host. There
had been no previous resource quota sets defined. Within 5 minutes, the
scheduler cpu usage on our dedicated 4-cpu qmaster/scheduler machine was
reduced from 100% of a processor (fairly typical behavior for the
current workload), to 2-20% of a processor and time from submission to
start for small jobs dropped from about 45 minutes to 15 to 30 seconds.
No other change in configuration or workload took place (to my
knowledge). The cluster context is fairly robust, with 2000 hosts, 12
queues, 10 parallel environments, and about 9000 queue instances. I
would be interested to know if any others have had similar experiences.
Geoscience Systems Development Advisor
One Allen Center
500 Dallas St.
Houston, Texas 77002
More information about the gridengine-users