[GE users] limit users * to slots=10

reuti reuti at staff.uni-marburg.de
Thu Dec 24 11:12:42 GMT 2009


Am 24.12.2009 um 01:27 schrieb marvnoo9:

>> BTW: Don't get confused between "slots" and "jobs". If you you want
>> to limit the number of jobs in an RQS (like maxujobs in the
>> scheduler), you have to define an additonal complex (named jobs for
>> sake of easiness, short jb) and set it consumable "JOB" with an
>> arbitrary high value set in `qconf -se global`, as the limit is in
>> the RQS.
>
> Reuti,
> Sorry but I don't understand ... wouldn't the `qconf -se global`  
> set a job limit applied to each execution host?

No, it's the availibility of a resource for the complete cluster,  
i.e. which are globally available with a fixed amount. Best example  
is the number of licenses you have for the complete cluster. These  
settings are not "copied" to each exec host, but will be used in  
addition to the one defined for each host (but the global one exist  
only one time).

Unfortunately there are no "cluster hosts" like "cluster queues"  
which would speed up the setup up by a common setting for all machines.


>   Even if arbitrarily high, I already have a slot limit on each  
> host which won't let new jobs go to the host if there are no more  
> slots.  My jobs limit needs to be per user for the entire cluster.
>
> But, now I'm even more confused...
> I want to able to set a limit on how many jobs (usually one slot  
> per job) a user can submit, but now with RQS have exceptions for  
> some users on some queues.  I've always used the maxujobs for that  
> (before RQS) but I guess that breaks down if a job requests a p.e.  
> that uses multiple slots?

Exactly this I wanted to point out: es long as you have serial jobs  
it's of course slots=jobs, but when you have parallel jobs which uses  
a PE this does no longer apply. Then "maxujobs" and "slots" will  
limit different things.


> Maybe what I really want is a slot limit and not rely on maxujobs???

This depends what you want to limit: the number jobs a user has or  
the number of slots his jobs use (or both: e.g. a user may run 2 jobs  
with 8 slots in total) - both makes sense and you have to decide.


>   Does that make sense?   I never considered limiting the number of  
> slots per user.  How does that interact with the 'slots' consumable  
> I see in 'qconf -sc' ?

Both must be fulfilled. I.e. the resource must be available (the  
consumable on an exechost) and its request may not exceed the imposed  
limits (in the RQS).


>   I have always just used 'qconf -ssconf' which has maxujobs but  
> nothing related to slots?

Yep.

-- Reuti


>
> Thanks for your help,
>    - Marvin
>
>
>
>> Hi,
>>
>> Am 20.12.2009 um 02:30 schrieb marvnoo9:
>>
>>> Thanks for pointing out the use of '{}'.
>>> I had missed that distinction in my first reading.
>>>
>>> I understand that the "first match" wins ... but what happens if
>>> there is no match in an RQS?
>>> Will it fall back to the global maxujobs in the sconf configuration?
>>
>> it should. Just try to confirm it ;-)
>>
>> But you can also include a last rule in each RQS which matches all,
>> to have it defined at a central point (the {*} does this already
>> unless limited by queue or project).
>>
>> BTW: Don't get confused between "slots" and "jobs". If you you want
>> to limit the number of jobs in an RQS (like maxujobs in the
>> scheduler), you have to define an additonal complex (named jobs for
>> sake of easiness, short jb) and set it consumable "JOB" with an
>> arbitrary high value set in `qconf -se global`, as the limit is in
>> the RQS.
>>
>> -- Reuti
>>
>>
>>> Thnaks again,
>>>    - Marv
>>>
>>>
>>>> Hi Marv,
>>>>
>>>> Resource Quotas are very very powerful and are worth digging  
>>>> into the
>>>> documentation for ...
>>>>
>>>> A few things ...
>>>>
>>>> This old wiki page has some additional examples:
>>>> http://wiki.gridengine.info/wiki/index.php/RQS_Common_Uses
>>>>
>>>> For your specific questions:
>>>>
>>>> There is a special grouping character "{}" that can be used to
>>>> define if
>>>> your quota limit applies in aggregate or to each individual  
>>>> member of
>>>> the scoped set.
>>>>
>>>> Example (hope I don't get this backwards in my coffeeless  
>>>> state ... )
>>>>
>>>> This would limit each individual user to no more than 10 slots:
>>>>
>>>>    limit users {*} to slots=10
>>>>
>>>> This would put a global limit for all users who can consume no
>>>> more than
>>>> 10 slots:
>>>>
>>>>    limit users * to slots=10
>>>>
>>>>
>>>> The above would be equiv to your maxujobs
>>>>
>>>>
>>>> And for the multiple queue question it's also pretty simple. You  
>>>> can
>>>> name queues in your quota limit line so you can have different
>>>> limits on
>>>> different queues. You can have multiple rules in effect - just
>>>> understand that the "first match" wins if there is any overlap.
>>>>
>>>>
>>>>
>>>>
>>>> marvnoo9 wrote:
>>>>> Pardon this simple question (I'm new to Resource Quotas) ...
>>>>> In http://wikis.sun.com/display/gridengine62u2/Managing+Resource
>>>>> +Quotas , I see the following:
>>>>> "For example, the following limit rule limits all users together
>>>>> to 10 slots: limit users * to slots=10."
>>>>>
>>>>> Does that really mean an aggregate limit of 10 slots?  The
>>>>> question of whether or not that means an aggregate limit is
>>>>> what's confusing me.
>>>>>
>>>>> How do I get the same functionality as what I now have for  
>>>>> maxujobs:
>>>>>
>>>>> qconf -ssconf | grep maxu
>>>>> maxujobs                          16
>>>>>
>>>>> ... but allow an exception for a single user or all individual
>>>>> users in a special queue to each be allowed more than the 16.
>>>>>
>>>
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do?
>>> dsForumId=38&dsMessageId=234319
>>>
>>> To unsubscribe from this discussion, e-mail: [users-
>>> unsubscribe at gridengine.sunsource.net].
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=234803
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=234840

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list