[GE users] Sequential array jobs

Charu Chaubal Charu.Chaubal at Sun.COM
Tue Sep 21 23:38:51 BST 2004



Orion Poplawski wrote:
> Charu Chaubal wrote:
> 
>> Orion Poplawski wrote:
>>  > Is it possible to submit an array job where the tasks execute 
>> sequentially?
>>  >
>>
>> Array jobs are intended to be mutually independent, so you can't have 
>> dependency on sequence among them.
>>
>> Why not just submit jobs one by one with -hold_jid?
>>
>> If you find that submit time is long for submitting many jobs at once, 
>> have you tried using GE 6?  Submit times for GE 6 can be quicker than 
>> for SGE 5.3.
> 
> 
> I guess I was just looking for a built in counter with the task ID 
> providing step of the job to complete.  Also, with -hold_jid I need to 
> determine the job ID of the previous qsub somehow, right?  Is there an 
> easy way to do this or do I need to parse the output from qsub?
> 

Parsing is not necessary --- you can use -hold_jid with job names.

So, you could do something like

qsub -N $name.1 -v COUNT=1 job.sh
x=1
while [ $x -lt $END ]
do
	prev=$name.$x
	x=x+step
	qsub -hold_jid $prev -N $name.$x -v COUNT=$x job.sh
done

(you could probably make this into a generic wrapper script, so you could have 
your users do something like 'qsub_seq <name> <start> <end> <step> <jobscript>')

Regards,
	Charu


> I think something like:
> 
>   qsub -t 1-10:sequential job
> 
> would be simpler than something like (excuse the shell mishmash):
> 
>   x=1
>   jid=`qsub job $x | <parse job id>`
>   x++
>   while [ $x -lt 11 ]
>   do
>      jid=`qsub -hold_jid $jid job $x | <parse job id>`
>      x=x+1
>   done
> 
> I am using GE 6.
> 
> Thanks!
> 

-- 
####################################################################
# Charu V. Chaubal              # Phone: (650) 786-7672 (x87672)   #
# Grid Computing Technologist   # Fax:   (650) 786-4591            #
# Sun Microsystems, Inc.        # Email: charu.chaubal at sun.com     #
####################################################################


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list