[GE users] Upper bound for array jobs?

Bernard Li bli at bcgsc.ca
Fri Aug 27 20:01:52 BST 2004


Hi Reuti:

> Can you please provide some more details: first you submit 
> one array job which creates all the 26,000 script files and 
> submits each script on it's own?

The following illustrates an example where a user submits an array job
with 100 tasks:

User creates a batchfile containing the commands to execute on the
cluster:

[test.batch]
/scratch/runtest1.sh
/scratch/runtest2.sh
/scratch/runtest3.sh
...
/scratch/runtest100.sh

User submits job to SGE using a script which parses the batch file:

mqsub --file test.batch --name test

mqsub creates individual script files to be submitted using qsub:

[test.1.sh] .. [test.100.sh]

#!/bin/sh

#$ -S /bin/sh

/scratch/runtest1.sh

mqsub also creates the array job script:

[test_array.sh]

#!/bin/sh

#$ -S /bin/sh

/scratch/test.$SGE_TASK_ID.sh

Finally mqsub executes the qsub command:

qsub -t 1-100 test_array.sh

I hope this makes sense.

Thanks,

Bernard

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list