[GE users] Limit load on NFS server

leinaddm ddm at bartol.udel.edu
Mon May 11 19:55:21 BST 2009


I have a cluster of about 30 compute nodes (~200 cores). The cluster has
8 NFS servers providing about 80TB of storage.

If a single user starts 200 jobs doing (heavy) IO, over the network, on
a single NFS server, it would not perform very well or it may even
crash. I am trying to devise a method to limit the number of jobs
accessing a single NFS server at once.

At the moment my idea would be to create a set of consumable complex
attributes, one for each nfs server, and have the users request one of
them when submitting jobs doing IO on a particular NFS server. In this
way the maximum number of jobs accessing at once a given NFS server can
be limited. 

I don't like this idea very much though, if the jobs are just doing IO
at the beginning of the script this approach would stop other jobs from
being executed even after the load on the nfs server is back to normal.

So I was looking at some better way to dynamically limit the load on the
NFS servers. Any suggestions?

I am running sge V61u4

Thanks, Daniel.


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list