[GE users] Re: Configure the number of simultaneous jobs in a queue

Neil Killeen nkilleen at unimelb.edu.au
Wed May 9 02:04:43 BST 2007


Ravi and others

many thanks, that worked (both GUI and qconf -mq).

Now I'd actually like to dynamically change the number of slots in  
the queue
with a cron job.

All I want to do is change the slots part of it.  Is it possible to  
do this with
a direct qconf command  or must one either

  - interactively edit
  - submit the full configuration file via qconf -Mq

My reading of the man page is that the latter is what I will need to do.
Is that correct ?


regards
Neil



> Neil
>
> On 08/05/2007, at 3:43 PM, Ravi Chandra Nallan wrote:
>
>> I am sorry if I confused you. The way you specify host specific  
>> configuration for the queue is by using the Queue "General  
>> Configuration Tab" that you started with.
>> The left pane you can find the "Attributes for Host/Hostgroup", it  
>> is here you select the host(/hostgroup) you have configured for  
>> the queue. Select it and modify the slots for this host specific  
>> cfg for the queue.
>> Equivalent in CLI,
>> $ qconf -sq <queuename>
>> you should see a [HOSTNAME=xx] for the slots attribute in your case.
>> use qconf -mq to modify the queue
>>
>> -Ravi
>>
>> Neil Killeen wrote:
>>> Ravi  (sorry for funny formatting below - I am trying to use the  
>>> digest list
>>>             but I'm not getting any emails, so I am cutting out  
>>> of the archive)
>>>
>>>
>>> thanks for the advice.  I did attempt to modify the specific  
>>> instance
>>> of the generic queue (via qmon).  However, via qmon, when I select
>>> 'Queue Control' and then the 'Queue Instances' TAB and then  
>>> select the
>>> actual queue, the 'Modify' button is greyed out so I can't modify  
>>> it.  Only
>>> the generic Cluster Queues TAB enables Modify
>>>
>>> WHat is the command-line equivalent to do this, with qconf I  
>>> assume ?
>>> I have been trawling through man qconf and queue_conf without
>>> great progress
>>>
>>> thanks
>>> Neil
>>>
>>>
>>>
>>> >>>>>
>>>
>>> From: Ravi Chandra Nallan <Ravichandra.Nallan at Sun.COM>
>>>
>>>
>>> Though the slots can be used to control the no. of running jobs in a
>>> queue, but it is the slot per queue instance than the slot per queue
>>> that we configure.
>>> And the slots per queue can be overridden from the actual queue
>>> configuration by using the specific queue instance configuration,
>>>
>>> For eg,
>>> a queue with hostlist > 1 , and slots =  1, will have: no. of  
>>> hosts *
>>> slots on total
>>>
>>> $ qconf -sq all.q | grep hostlist
>>> hostlist              @allhosts
>>>
>>> $ qconf -shgrp @allhosts
>>> group_name @allhosts
>>> hostlist host1 host2
>>>
>>> $ qconf -sq all.q | grep slots
>>> slots                 1,[host1=2]
>>>
>>> $ qstat -g c
>>> CLUSTER QUEUE                   CQLOAD   USED  AVAIL  TOTAL  
>>> aoACDS  cdsuE
>>> -------------------------------------------------------------------- 
>>> -----------
>>> all.q                             0.00      0      3      3       
>>> 0      0
>>>
>>> here the 3 slots = host1=2 + host2=1 (from queue's default  
>>> configuration)
>>>
>>> Neil, the slots you are seeing in the global configuration is the
>>> default queue configuration, if you can notice the hostname below  
>>> the
>>> standard "@/", select it. This is the host(/hostgroup) specific
>>> configuration which would have been set to 64 while installation.  
>>> Hence
>>> you should be able to run 64 *simultaneous* jobs.
>>> If you want to control the no. of simultaneous jobs, set the  
>>> slots in
>>> the host specific conf, or remove the host specific conf and use the
>>> global queue conf for controlling.
>>>
>>> Check the explanation for hostlist value in the man queue_conf(5),
>>> -Ravi
>>>
>>> Rayson Ho wrote:
>>> > Yes, you can use "slots" to control the number of running jobs  
>>> in a
>>> > queue.
>>> >
>>> > Are you sure there are more *running* jobs than the number of
>>> > available slots?? What did you find when you ran qstat when  
>>> there are
>>> > more than 64 jobs submitted??
>>> >
>>> > Rayson
>>> >
>>> >
>>> > On 5/6/07, Neil Killeen <nkilleen at unimelb.edu.au> wrote:
>>> >> Hi
>>> >>
>>> >> I have version  6.0u10 (default installation) operating on an  
>>> Altix
>>> >> multi-processor (N=64)   machine.
>>> >>
>>> >> I have one queue.  I want to configure that queue to set the  
>>> maximum
>>> >> number of
>>> >> simultaneous jobs the queue will run (independent of users and
>>> >> scheduling).
>>> >>
>>> >> Using the QMON GUI I click on 'Queue Control',  and select the
>>> >> 'CLuser Queue'
>>> >> TAB.  This shows my one queue (all.q) with columns 'AVAIL=64' and
>>> >> 'TOTAL=64'
>>> >> I suspect the 64s came from the install script finding out how  
>>> many
>>> >> CPUs the
>>> >> system has (64).
>>> >>
>>> >> I select my one CLuster Queue and then click 'modify'.   
>>> Selecting the
>>> >> 'General Configuration' TAB  I see
>>> >> a property called 'Slots'  It has the value 1.  The manual  
>>> suggests
>>> >> this is the property I want.
>>> >>
>>> >> However, I find that I can run an infinite number of simultaneous
>>> >> jobs on my queue (just by submitting with 'qsub').   CHanging the
>>> >> 'Slots' parameter
>>> >> has no impact on this.
>>> >>
>>> >>
>>> >> 1. What exactly are the Cluster queue columns 'AVail' and  
>>> 'Total' and
>>> >> how were they populated ?
>>> >> 2. What is slots exactly ?
>>> >> 3. How do I constrain the number of simultaneous jobs on my  
>>> queue to N ?
>>> >> 4. If 'slots' is the answer, why did it have no effect when I  
>>> changed
>>> >> it ?
>>> >>
>>> >>
>>> >> I have looked at the FAQs, HowTos and Manuals without finding the
>>> >> answer (if it's not slots)
>>> >>
>>> >>
>>> >>
>>> >> many thanks
>>> >> Neil
>>> >>
>>> >>  
>>> -------------------------------------------------------------------- 
>>> -
>>> >> To unsubscribe, e-mail: users- 
>>> unsubscribe at gridengine.sunsource.net
>>> >> For additional commands, e-mail: users- 
>>> help at gridengine.sunsource.net
>>> >>
>>> >>
>>> >
>>> >  
>>> -------------------------------------------------------------------- 
>>> -
>>> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>> > For additional commands, e-mail: users- 
>>> help at gridengine.sunsource.net
>>> >
>>>
>>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list