[GE users] Array jobs with each job in a separate host?

Reuti reuti at staff.uni-marburg.de
Fri Aug 4 11:16:23 BST 2006


Hi,

Am 04.08.2006 um 02:49 schrieb Borja Sotomayor:

> Hi everyone,
>
> I am running into the following problem: I want to submit an array  
> job in such a way that each task runs on a separate host. I realize  
> one possible solution is to set the number of slots to 1, or use a  
> solution along the lines of this:
>
> http://gridengine.info/articles/2005/09/22/allowing-user-jobs-to- 
> take-over-entire-nodes
>
> However, in my case, I still want to allow other array jobs to run  
> at the same time. e.g. If I have four nodes {A,B,C,D} and I submit  
> an array job with 3 tasks which end up in {A,B,C}, and then submit  
> the same array job with 2 tasks, I want to allow one of the tasks  
> to be scheduled in those three nodes (instead of having to wait for  
> them to become available).
>
> In other words, I want to guarantee exclusive use of a host  
> *inside* an array job, but not across other jobs. Is this possible?

how many users want this distribution scheme? I could think of  
something like a "job-license", an integer complex which has a  
consumable value of 1 for each node (qconf -me <node>). However, the  
default request in the complex definition is 0, so it won't matter  
for other users.

You would have to request this "-l job_license_a=1", and you can only  
get one slot on each machine, after you attached it to all machines.  
For your next job you need then a job_license_b, if you on your own  
have already a job running in the cluster, and want another array job  
to run parallel to the other on the nodes. Other users don't have to  
request anything. If this scheme is just for one or two users, it  
might be worth to implement it this way.

BTW: For our own usage, I just made a small change in our SGE to  
allow also "empty integer values", i.e. "-l job_license_a" will mean   
"-l job_license_a=1", as it is already implemented for boolean  
complexes where it defaults to "TRUE" if left out. Is this also  
needed by others?

Cheers - Reuti

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list