[GE users] Sequential array jobs

Stephan Grell - Sun Germany - SSG - Software Engineer stephan.grell at sun.com
Wed Sep 22 10:17:00 BST 2004


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Andy Schwierskott wrote:

> Hi,
>
> job dependencies on job names are a 6.0 feature. 


Well, we have a limited job dependency functionality in 5.3. It got a 
lot more
usefull in 6.0. One can use regular expressions in the dependency 
request and
job names do not have to be unique anymore.

Stephan

>
>
> Andy
>
> On Tue, 21 Sep 2004, Charu Chaubal wrote:
>
>>
>>
>> Orion Poplawski wrote:
>>
>>> Charu Chaubal wrote:
>>>
>>>> Orion Poplawski wrote:
>>>>  > Is it possible to submit an array job where the tasks execute 
>>>> sequentially?
>>>>  >
>>>>
>>>> Array jobs are intended to be mutually independent, so you can't 
>>>> have dependency on sequence among them.
>>>>
>>>> Why not just submit jobs one by one with -hold_jid?
>>>>
>>>> If you find that submit time is long for submitting many jobs at 
>>>> once, have you tried using GE 6?  Submit times for GE 6 can be 
>>>> quicker than for SGE 5.3.
>>>
>>>
>>>
>>> I guess I was just looking for a built in counter with the task ID 
>>> providing step of the job to complete.  Also, with -hold_jid I need 
>>> to determine the job ID of the previous qsub somehow, right?  Is 
>>> there an easy way to do this or do I need to parse the output from 
>>> qsub?
>>>
>>
>> Parsing is not necessary --- you can use -hold_jid with job names.
>>
>> So, you could do something like
>>
>> qsub -N $name.1 -v COUNT=1 job.sh
>> x=1
>> while [ $x -lt $END ]
>> do
>>     prev=$name.$x
>>     x=x+step
>>     qsub -hold_jid $prev -N $name.$x -v COUNT=$x job.sh
>> done
>>
>> (you could probably make this into a generic wrapper script, so you 
>> could have your users do something like 'qsub_seq <name> <start> 
>> <end> <step> <jobscript>')
>>
>> Regards,
>>     Charu
>>
>>
>>> I think something like:
>>>
>>>   qsub -t 1-10:sequential job
>>>
>>> would be simpler than something like (excuse the shell mishmash):
>>>
>>>   x=1
>>>   jid=`qsub job $x | <parse job id>`
>>>   x++
>>>   while [ $x -lt 11 ]
>>>   do
>>>      jid=`qsub -hold_jid $jid job $x | <parse job id>`
>>>      x=x+1
>>>   done
>>>
>>> I am using GE 6.
>>>
>>> Thanks!
>>>
>>
>>
>
>
> Regards,
> Mit freundlichen Gruessen,
> Andy
> Schwierskott
>
> -- 
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> Andy Schwierskott           Tel:     +49 941 3075-200  (x60200)
> N1 Grid Engine Engineering  Support: +49 941 3075-250  (x60250)
> Sun Microsystems GmbH       Fax:     +49 941 3075-222  (x60222)
> Dr.-Leo-Ritter-Str. 7       mailto:andy.schwierskott at sun.com
> D-93049 Regensburg          http://www.sun.com/gridware
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list