[GE users] Array jobs - Dynamic hostname resource?

John Tseng jtseng at montalvosystems.com
Sat May 26 16:01:28 BST 2007


    [ The following text is in the "UTF-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

The scheduler sees the same "request" for all tasks


If you really wanted round robin around three hosts,  could you just 
submit three task jobs?

qsub -t 1-100:3 -l hostname=host1 myrun
qsub -t 2-100:3 -l hostname=host2 myrun
qsub -t 3-100:3 -l hostname=host3 myrun

Or the more interesting
qsub -t 1-100:3 -q '*@@hostgroup1' myrun
qsub -t 2-100:3 -q '*@@hostgroup2' myrun
qsub -t 3-100:3 -q '*@@hostgroup3' myrun


A complicated possibility would be to have a complex under 
job_load_adjustment in the scheduler to artificially increase the "load" 
to  make a host "unavailable", but this is not a static value as it 
decays over time.  You would need to set this as a alarm threshold to 
make it "unavailable".    This is hard to debug and requires 
administrative overhead for every job type as well as a separate cluster 
queue.


-john


Daire Byrne wrote:
> Hi,
>
> As you can only set a common overall hard resource for all tasks can anybody think of a good way to specify individual hosts for each task? ie something like:
>
>   qsub << END
>   #!/bin/sh
>
>   SERVERS=( host1 host2 host3 )
>   ((SERVER_INDEX = \$SGE_TASK_ID - 1))
>
>   #\$ -l hostname=\${SERVERS_BACKUP[\$SERVER_INDEX]}
>
>   echo hostname
>
> I know the above won't work as the -l stuff is only interpreted at submit time but hopefully it illustrates what I'm trying to do. I'd prefer use array tasks but if its not possible I can probably use individual jobs.
>
> Cheers,
>
> Daire
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list