[GE users] Array jobs - Dynamic hostname resource?

Reuti reuti at staff.uni-marburg.de
Sat May 26 17:35:55 BST 2007


Daire:

Am 26.05.2007 um 17:01 schrieb John Tseng:

> The scheduler sees the same "request" for all tasks
>
>
> If you really wanted round robin around three hosts,  could you  
> just submit three task jobs?
>
> qsub -t 1-100:3 -l hostname=host1 myrun
> qsub -t 2-100:3 -l hostname=host2 myrun
> qsub -t 3-100:3 -l hostname=host3 myrun
>
> Or the more interesting
> qsub -t 1-100:3 -q '*@@hostgroup1' myrun
> qsub -t 2-100:3 -q '*@@hostgroup2' myrun
> qsub -t 3-100:3 -q '*@@hostgroup3' myrun
>
>
> A complicated possibility would be to have a complex under  
> job_load_adjustment in the scheduler to artificially increase the  
> "load" to  make a host "unavailable", but this is not a static  
> value as it decays over time.  You would need to set this as a  
> alarm threshold to make it "unavailable".    This is hard to debug  
> and requires administrative overhead for every job type as well as  
> a separate cluster queue.

maybe the question is more: what do you want to achieve in the end,  
as SGE will select proper machines for the job. So why specify them  
at all? If you want simply a job to be run on all machines, then a PE  
with allocation_rule set to the fixed value 1 would also do (or a  
queue with one slot fro this PE) and submit a parallel job which uses  
qrsh in a loop to reach all nodes.

-- Reuti

>
> -john
>
>
> Daire Byrne wrote:
>> Hi,
>>
>> As you can only set a common overall hard resource for all tasks  
>> can anybody think of a good way to specify individual hosts for  
>> each task? ie something like:
>>
>>   qsub << END
>>   #!/bin/sh
>>
>>   SERVERS=( host1 host2 host3 )
>>   ((SERVER_INDEX = \$SGE_TASK_ID - 1))
>>
>>   #\$ -l hostname=\${SERVERS_BACKUP[\$SERVER_INDEX]}
>>
>>   echo hostname
>>
>> I know the above won't work as the -l stuff is only interpreted at  
>> submit time but hopefully it illustrates what I'm trying to do.  
>> I'd prefer use array tasks but if its not possible I can probably  
>> use individual jobs.
>>
>> Cheers,
>>
>> Daire
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list