[GE users] manage NFS resources

markhewitt mh613 at york.ac.uk
Wed Sep 9 09:56:54 BST 2009


murple wrote:
> Hi,
> 
> I'm not sure if it's possible but I'm looking for a good solution for 
> following problem. One type of task running in our cluster starts a lot 
> of identical jobs. All of them need to read some big input files from 
> the same NFS fileserver to start.
> So in the beginning they all wait for I/O. Better would be to delay the 
> start of additional jobs until bandwidth is available again.  One 
> possibility is to script the starttime of the jobs accordingly. But for 
> this one needs to guess the time needed for a single job to read the input.
> 
> I thing a consumable is not a solution since this would only be freed 
> after job ends (long time after network bandwidth is freed).
> 
> This week we had one occasion where almost 100% of CPU was in IOWAIT 
> (according to Ganglia) for about 2.5 hours.
> 
> regards, Andreas

Don't know the exact syntax but how about an array job where the 
parameter it takes is used as a multiplier for 'sleep'. All jobs would 
enter SGE at the same time but most would not do anything for a period 
of time before waking up.

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=216530

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list