[GE users] manage NFS resources

markhewitt mh613 at york.ac.uk
Wed Sep 9 09:56:54 BST 2009

murple wrote:
> Hi,
> I'm not sure if it's possible but I'm looking for a good solution for 
> following problem. One type of task running in our cluster starts a lot 
> of identical jobs. All of them need to read some big input files from 
> the same NFS fileserver to start.
> So in the beginning they all wait for I/O. Better would be to delay the 
> start of additional jobs until bandwidth is available again.  One 
> possibility is to script the starttime of the jobs accordingly. But for 
> this one needs to guess the time needed for a single job to read the input.
> I thing a consumable is not a solution since this would only be freed 
> after job ends (long time after network bandwidth is freed).
> This week we had one occasion where almost 100% of CPU was in IOWAIT 
> (according to Ganglia) for about 2.5 hours.
> regards, Andreas

Don't know the exact syntax but how about an array job where the 
parameter it takes is used as a multiplier for 'sleep'. All jobs would 
enter SGE at the same time but most would not do anything for a period 
of time before waking up.


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list