[GE users] queue seq num and PE wildcards

reuti reuti at staff.uni-marburg.de
Sun Aug 22 13:47:25 BST 2010

Am 21.08.2010 um 23:37 schrieb gragghia:

> The example that I showed was on a simple test installation.  Our 
> production installation uses $fill_up for the MPI PEs.  We have one PE 
> per Infiniband fabric in order to ensure that an MPI job doesn't get 
> assigned to nodes on more than one fabric (we have four).

Yes, this is the way to go for now.

>  I expected 
> the jobs to follow the sequence number of the queues (since no other 
> features directly address the order in which resources are 
> preferentially assigned).
> We have good luck in using multiple PEs for our IB fabrics and 
> submitting jobs with "-pe mpi* ##", but this is preventing use of the 
> queue sequence feature.  If there is another option to ensure that a job 
> doesn't use multiple queues at once (without making hard resource 
> requests), then I'd certainly be interested in hearing how to do it.

Although I on my own suggested to use multiple queues in such a setup in the past, you can also use hostgroups and have only one queue:

$ qconf -sq all.q
seq_no 0,[@group1=10],[@group2=20],[@group3=30],[@group4=40]
pe_list NONE,[@group1=mpi1],[@group2=mpi2],[@group3=mpi3],[@group4=mpi4]

Does such a setup help?

-- Reuti

> - Geri
>> Do you expect the mpi* to follow a sequence number of PEs (which doesn't exist), or inside an already chosen PE to use the sequence number of the queue instances? What's your PE setup? It looks like $pe_slots as allocation_rule in your setup.
>> -- Reuti
> -- 
> Gerald Ragghianti
> Newton HPC Program http://newton.utk.edu/
> Office of Information Technology
>  Research Computing Support
>  Professional Technical Services
> The University of Tennessee
> 2309 Kingston Pike
> Knoxville, TN 37996
> Phone: 865-974-2448
> /-------------------------------------\
> | One Contact       OIT: 865-974-9900 |
> | Many Solutions         help.utk.edu |
> \-------------------------------------/
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=275846
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list