[GE users] Philosophy of queue setup

Reuti reuti at staff.uni-marburg.de
Wed Nov 30 16:27:07 GMT 2005


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Hi Jean-Paul,

Am 30.11.2005 um 13:36 schrieb Jean-Paul Minet:

> Hi,
>
> We have a cluster of 128 bi-proc nodes with 4 GB RAM each, and a  
> couple of SMP boxes (8 CPU, 32 GB RAM).  Users from several  
> research entities will submit sequential jobs as well as MPI and  
> OpenMP jobs. We would like to setup SGE so that:
>
> - sequential jobs requiring less than 4 Gb run on the bi-proc  
> nodes, while jobs requiring over 4 Gb run on one of the SMP boxes;

well, jobs with more than 4 GB will of course never run on the  
smaller nodes, but to specify a minimum for the SMP machines might  
lead to empty machines, although still jobs are waiting. Maybe giving  
the sequential queue instances on the SMP machines only 4 slots can  
be a compromise and still let some room for parallel jobs to start  
immediately (if you have another queue for them).

> - parallel OpenMP jobs run on the SMP boxes (with a max of 8 slots  
> per job);
> - parallel MPI jobs run on the bi-proc nodes by default (with a max  
> of 48 slots), but could be willingly directed by the users to the  
> SMP boxes when required (and if requesting less than the total  
> number of CPUs of the SMP boxes).

what about attaching a special complex (resource) to the SMP slots in  
the queue definition of type BOOL/forced? If the user requests this  
resource, the job will go only to these nodes (and s/he will never  
get a mixture of slots).

> I have setup an OMP parallel environment with 8 slots, as well as  
> an mpich parallel environment with 48 slots.  From there, I have no  
> clear idea

the specified slots are the one in total in this PE. So this would  
need to be 8 * SMP-nodes for the OMP (allocation rule $pe_slots) and  
256 for the mpich, unless it was really your intention to use only  
part of the cluster for the parallel jobs.

> on how best to structure the queues (and their attached PE) to get  
> the desired behavior.  Also, how to prevent job requiring *less*  
> memory than a specified amount to run on specific (SMP) hosts?

Cheers - Reuti

> Any help will be appreciated
>
> Rgds
>
> Jean-Paul
> -- 
> Jean-Paul Minet
> Gestionnaire CISM - Institut de Calcul Intensif et de Stockage de  
> Masse
> Université Catholique de Louvain
> Tel: (32) (0)10.47.35.67 - Fax: (32) (0)10.47.34.52
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list