[GE users] Philosophy of queue setup

Reuti reuti at staff.uni-marburg.de
Tue Dec 6 12:02:32 GMT 2005


Hi,

Am 06.12.2005 um 11:05 schrieb Jean-Paul Minet:

> Reuti,
>
>>> - sequential jobs requiring less than 4 Gb run on the bi-proc   
>>> nodes, while jobs requiring over 4 Gb run on one of the SMP boxes;
>> well, jobs with more than 4 GB will of course never run on the   
>> smaller nodes, but to specify a minimum for the SMP machines  
>> might  lead to empty machines, although still jobs are waiting.  
>> Maybe giving  the
>
> true, but we really want to keep the SMP machines available for SMP  
> jobs and/or jobs requiring lots or RAM.  There are enough small  
> nodes to satisfy other requests.  So, how, technically, would we  
> prevent small memory jobs (< 4 GB RAM) to run on SMP boxes (and  
> still manage memory requests as not to oversubscribe the available  
> 32 GB RAM)?

Okay.

As we here don't need hard limits for our jobs, I make the complex  
virtual_free consumable and set it on each host (qconf -me  
<hostname>") to the amount of built in memory minus 100M. You can  
also (like with your mem_free) make it forced and/or make mem_free  
consumable and set it to this value. Internally SGE will use the  
lower value of a) the consumable or b) the real load value.

>>> - parallel OpenMP jobs run on the SMP boxes (with a max of 8  
>>> slots  per job);
>>> - parallel MPI jobs run on the bi-proc nodes by default (with a  
>>> max  of 48 slots), but could be willingly directed by the users  
>>> to the  SMP boxes when required (and if requesting less than the  
>>> total  number of CPUs of the SMP boxes).
>> what about attaching a special complex (resource) to the SMP slots  
>> in  the queue definition of type BOOL/forced? If the user requests  
>> this  resource, the job will go only to these nodes (and s/he will  
>> never  get a mixture of slots).
>
> What I did up to now is to define mem_free and num_proc as forced  
> requestable resources (together with two PE's: mpich and smp).  In  
> this way, users request themselves the platform on which they will  
> run (biproc or octoproc) and SGE can manage memory usage and avoid  
> oversubscription.  Maybe there is a more clever/flexible way to  
> handle this...

Personally I would have defined two hostgroups @biproc and @octoproc  
and two BOOL complexes "biproc" and "octopro" (both forced). These  
complexes could be attached to the hostgroups in the queue  
definition. Then the users could use "-l biproc". But it's just  
personal taste as stated, if you are happy with num_proc it's the  
same in the end I guess. - Reuti

> Thanks & rgds
>
> Jean-paul
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list