[GE users] limiting the number of jobs scheduled per cycle

Craig Tierney ctierney at HPTI.com
Thu May 5 19:00:09 BST 2005


On Wed, 2005-05-04 at 14:19, Shane Canon wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> This is a repeat of a similar question from another person...
> 
> http://gridengine.sunsource.net/servlets/ReadMsg?list=users&msgId=24885
> 
> I couldn't find a response.
> 
> We are seeing a similar issue.  We would like to limit the number of
> jobs that get launched together during a scheduling cycle on a per user
> basis.  I'm trying to create a method using standard resources.  I was
> thinking of have the jobs modify there own resource requirement
> (consumption) after they start running, but I don't think qalter allows
> this.  Another method would be for the job to modify the quantity for a
> global resource.  I think I could make this work, but it would be rather
> convoluted.
> 
> Any ideas how we can accomplish this?

If you do not schedule many jobs (< 10 a minute) and your system isn't
very big,  you may try changing the scheduler to schedule jobs
immediately so that you don't get spikes every N seconds.

Also, modify your autofs configuration so that the mounts do not
timeout.  This won't fix the problem when you start the system, but
it will prevent mounts from going away and having to remount later.
This is a trick we figured out 5+ years ago on our clusters and we have
continued it.  

Are you using a commercial version of Redhat or some rebuilds?
If you are paying for support, get them to fix it.  If not, switch
to a stock kernel.  For clusters, we have found that stock kernels work
better (YMMV).  It does mess with the support model though.

Craig


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list