[GE users] job context and array tasks

Andy Schwierskott andy.schwierskott at sun.com
Tue Nov 6 14:09:29 GMT 2007


Hi Mark,

> Hi Andy,
>
>>    - have a handy syntax enhancement to set context variables for
>> tasks and
>>      ranges of tasks or all tasks, e.g.
>>
>>      qsub/qalter -Xc [flexible_range_specifier=]variable=value
>>
>>      where "flexible_range_specifier" at least should be the current
>>      "task_id_range" (from the qsub syntax), but probably even a bit
>> more
>>      like support for adressing individual tasks:
>>
>>          2,3,6-12:2,18,100:4=restart=FALSE
>
>
> A more flexible range specification would be also really useful for parallel
> environments. For some calculations we need specialized domain
> decomposition. We might, for example, decompose the domain into 12, 8, 6, 4
> subdomains.  In which case one can't simply specify "-pe foo 4-12" and hope
> for the best, but a finer grained control is needed:
>  Eg, "-pe foo 4,6,8,12".
>
> Of course, this only makes sense when we can simultaneously specify masterq
> resources too.

and some more flexibility with wildcard PEs and ranges. :-)

A customer told me that he would like to be able to ensure that when a range
is requested for a PE job which can go to different machines types on
machine type A not more than 16 slots should be granted but on machine type
B up to 32 slots may be granted. Likely the only clean solution would be to
offer a true logical operation support for any types of resource requests:

    qsub "-pe sw1_machA 4-16 | -pe sw1_machB 4-32"

Probably a more easy to implement solution would be to support a range
specification in the PE configuration

    qconf -sp sw1_machA
    [...]
    slot_range 4-16

    qconf -sp sw1_machB
    [...]
    slot_range 4-32

and the do a wildcard submission:

    qsub -pe sw1_mach* 4-32

Nevertheless that's now really a different story than the original question
about context variables.

Andy

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list