[GE users] Allow users to limit concurrent array job task execution

reuti reuti at staff.uni-marburg.de
Thu Nov 19 10:47:26 GMT 2009


Am 18.11.2009 um 15:51 schrieb heywood:

> Is it possible to limit the number of running *jobs*? Not tasks of  
> a job
> array, but letting the user submit a lot of jobs but specify a maximum
> number that can run at once?

First let me add to the original issue, that there is also the global  
parameter "max_aj_instances" in `qconf -monf`which will control the  
number of concurrent array job tasks respectively defines an upper  
limit for the -tc (although no error will be output when you specify  
a higher value there).

==

In your case there is a parameter in the scheduler configuration  
`qconf -msconf` called "maxujobs" which also defines it at a global  
level the number of each user's *running* jobs.

Starting with SGE 6.2u3 you can define different limits per user: we  
need a custom complex e.g. called "jobs" which must be set to "type  
INT", "requestable YES"*, "consumable JOB" and "default 1".

Second thing is to define a global arbitrary high value for this  
resource (it also serves as the total number of jobs you allow in the  
cluster) in the global exechost configuration `qconf -me global`.

Now we can setup an RQS, which contains limits for individual users  
or ACLs:

limit users foo to jobs=10
limit users {@bar} to jobs=5

-- Reuti

*) Maybe used to bypass the limit when one requests jobs=0, but there  
is a bug which will prevent the jobs from being scheduled when it's  
set to "NO" right now.


> Todd
>
>
> On 11/18/09 4:38 AM, "aja" <alena.plestilova at sun.com> wrote:
>
>> Hi,
>>
>> this feature is not documented in 6.2u4. But it's already added to  
>> 6.2u5.
>> Here is the man qsub input for this feature:
>>
>>      -tc max_running_tasks
>>           -allow users to limit concurrent array job task  execu-
>>           tion.  Parameter  max_running_tasks  specifies  maximum
>>           number of simultaneously running tasks.  For example we
>>           have  running  SGE  with 10 free slots. We call qsub -t
>>           1-100 -tc 2  jobscript.  Then  only  2  tasks  will  be
>>           scheduled to run even when 8 slots are free.
>>
>> Regards,
>> aja
>>
>>
>> On 11/17/09 19:06, svibhore wrote:
>>> Thanks Reuti, this is great!
>>>
>>>
>>> On Tue, Nov 17, 2009 at 9:12 PM, reuti <reuti at staff.uni-marburg.de
>>> <mailto:reuti at staff.uni-marburg.de>> wrote:
>>>
>>>     Am 17.11.2009 um 15:40 schrieb svibhore:
>>>
>>>> Hello,
>>>>
>>>> Release notes for 6.2u4 has a entry for job throttle.
>>>>
>>>>
>>>> Bugs fixed in SGE 6.2u4 since release 6.2u3
>>>> -------------------------------------------
>>>>
>>>> 2542 6684887 allow users to limit concurrent array job task
>>>     execution
>>>>
>>>> I do not see any documentation related to this feature, can anyone
>>>> point me in right direction.
>>>
>>>     In issue 2542 the syntax is mentioned:
>>>
>>>     $ qsub -t 1-20 -tc 5 test.sh
>>>
>>>     to limit it to e.g. 5 and AFAICS it's working - it's just  
>>> missing
>>>     from the man page. Can you please file a documentation issue?
>>>
>>>     -- Reuti
>>>
>>>
>>>>
>>>> Regards,
>>>> Shantanu
>>>
>>>     ------------------------------------------------------
>>>
>>> http://gridengine.sunsource.net/ds/viewMessage.do? 
>>> dsForumId=38&dsMessageId=22
>>> 7467
>>>
>>> <http://gridengine.sunsource.net/ds/viewMessage.do? 
>>> dsForumId=38&dsMessageId=2
>>> 27467>
>>>
>>>     To unsubscribe from this discussion, e-mail:
>>>     [users-unsubscribe at gridengine.sunsource.net
>>>     <mailto:users-unsubscribe at gridengine.sunsource.net>].
>>>
>>>
>>
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=227689
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=227889

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list