[GE users] Re: Configure the number of simultaneous jobs in a queue

Rayson Ho rayrayson at gmail.com
Wed May 9 02:51:37 BST 2007


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

See scripting HOWTO:

http://gridengine.sunsource.net/howto/scripting.html

Rayson



On 5/8/07, Neil Killeen <nkilleen at unimelb.edu.au> wrote:
> Ravi and others
>
> many thanks, that worked (both GUI and qconf -mq).
>
> Now I'd actually like to dynamically change the number of slots in
> the queue
> with a cron job.
>
> All I want to do is change the slots part of it.  Is it possible to
> do this with
> a direct qconf command  or must one either
>
>  - interactively edit
>  - submit the full configuration file via qconf -Mq
>
> My reading of the man page is that the latter is what I will need to do.
> Is that correct ?
>
>
> regards
> Neil
>
>
>
> > Neil
> >
> > On 08/05/2007, at 3:43 PM, Ravi Chandra Nallan wrote:
> >
> >> I am sorry if I confused you. The way you specify host specific
> >> configuration for the queue is by using the Queue "General
> >> Configuration Tab" that you started with.
> >> The left pane you can find the "Attributes for Host/Hostgroup", it
> >> is here you select the host(/hostgroup) you have configured for
> >> the queue. Select it and modify the slots for this host specific
> >> cfg for the queue.
> >> Equivalent in CLI,
> >> $ qconf -sq <queuename>
> >> you should see a [HOSTNAME=xx] for the slots attribute in your case.
> >> use qconf -mq to modify the queue
> >>
> >> -Ravi
> >>
> >> Neil Killeen wrote:
> >>> Ravi  (sorry for funny formatting below - I am trying to use the
> >>> digest list
> >>>             but I'm not getting any emails, so I am cutting out
> >>> of the archive)
> >>>
> >>>
> >>> thanks for the advice.  I did attempt to modify the specific
> >>> instance
> >>> of the generic queue (via qmon).  However, via qmon, when I select
> >>> 'Queue Control' and then the 'Queue Instances' TAB and then
> >>> select the
> >>> actual queue, the 'Modify' button is greyed out so I can't modify
> >>> it.  Only
> >>> the generic Cluster Queues TAB enables Modify
> >>>
> >>> WHat is the command-line equivalent to do this, with qconf I
> >>> assume ?
> >>> I have been trawling through man qconf and queue_conf without
> >>> great progress
> >>>
> >>> thanks
> >>> Neil
> >>>
> >>>
> >>>
> >>> >>>>>
> >>>
> >>> From: Ravi Chandra Nallan <Ravichandra.Nallan at Sun.COM>
> >>>
> >>>
> >>> Though the slots can be used to control the no. of running jobs in a
> >>> queue, but it is the slot per queue instance than the slot per queue
> >>> that we configure.
> >>> And the slots per queue can be overridden from the actual queue
> >>> configuration by using the specific queue instance configuration,
> >>>
> >>> For eg,
> >>> a queue with hostlist > 1 , and slots =  1, will have: no. of
> >>> hosts *
> >>> slots on total
> >>>
> >>> $ qconf -sq all.q | grep hostlist
> >>> hostlist              @allhosts
> >>>
> >>> $ qconf -shgrp @allhosts
> >>> group_name @allhosts
> >>> hostlist host1 host2
> >>>
> >>> $ qconf -sq all.q | grep slots
> >>> slots                 1,[host1=2]
> >>>
> >>> $ qstat -g c
> >>> CLUSTER QUEUE                   CQLOAD   USED  AVAIL  TOTAL
> >>> aoACDS  cdsuE
> >>> --------------------------------------------------------------------
> >>> -----------
> >>> all.q                             0.00      0      3      3
> >>> 0      0
> >>>
> >>> here the 3 slots = host1=2 + host2=1 (from queue's default
> >>> configuration)
> >>>
> >>> Neil, the slots you are seeing in the global configuration is the
> >>> default queue configuration, if you can notice the hostname below
> >>> the
> >>> standard "@/", select it. This is the host(/hostgroup) specific
> >>> configuration which would have been set to 64 while installation.
> >>> Hence
> >>> you should be able to run 64 *simultaneous* jobs.
> >>> If you want to control the no. of simultaneous jobs, set the
> >>> slots in
> >>> the host specific conf, or remove the host specific conf and use the
> >>> global queue conf for controlling.
> >>>
> >>> Check the explanation for hostlist value in the man queue_conf(5),
> >>> -Ravi
> >>>
> >>> Rayson Ho wrote:
> >>> > Yes, you can use "slots" to control the number of running jobs
> >>> in a
> >>> > queue.
> >>> >
> >>> > Are you sure there are more *running* jobs than the number of
> >>> > available slots?? What did you find when you ran qstat when
> >>> there are
> >>> > more than 64 jobs submitted??
> >>> >
> >>> > Rayson
> >>> >
> >>> >
> >>> > On 5/6/07, Neil Killeen <nkilleen at unimelb.edu.au> wrote:
> >>> >> Hi
> >>> >>
> >>> >> I have version  6.0u10 (default installation) operating on an
> >>> Altix
> >>> >> multi-processor (N=64)   machine.
> >>> >>
> >>> >> I have one queue.  I want to configure that queue to set the
> >>> maximum
> >>> >> number of
> >>> >> simultaneous jobs the queue will run (independent of users and
> >>> >> scheduling).
> >>> >>
> >>> >> Using the QMON GUI I click on 'Queue Control',  and select the
> >>> >> 'CLuser Queue'
> >>> >> TAB.  This shows my one queue (all.q) with columns 'AVAIL=64' and
> >>> >> 'TOTAL=64'
> >>> >> I suspect the 64s came from the install script finding out how
> >>> many
> >>> >> CPUs the
> >>> >> system has (64).
> >>> >>
> >>> >> I select my one CLuster Queue and then click 'modify'.
> >>> Selecting the
> >>> >> 'General Configuration' TAB  I see
> >>> >> a property called 'Slots'  It has the value 1.  The manual
> >>> suggests
> >>> >> this is the property I want.
> >>> >>
> >>> >> However, I find that I can run an infinite number of simultaneous
> >>> >> jobs on my queue (just by submitting with 'qsub').   CHanging the
> >>> >> 'Slots' parameter
> >>> >> has no impact on this.
> >>> >>
> >>> >>
> >>> >> 1. What exactly are the Cluster queue columns 'AVail' and
> >>> 'Total' and
> >>> >> how were they populated ?
> >>> >> 2. What is slots exactly ?
> >>> >> 3. How do I constrain the number of simultaneous jobs on my
> >>> queue to N ?
> >>> >> 4. If 'slots' is the answer, why did it have no effect when I
> >>> changed
> >>> >> it ?
> >>> >>
> >>> >>
> >>> >> I have looked at the FAQs, HowTos and Manuals without finding the
> >>> >> answer (if it's not slots)
> >>> >>
> >>> >>
> >>> >>
> >>> >> many thanks
> >>> >> Neil
> >>> >>
> >>> >>
> >>> --------------------------------------------------------------------
> >>> -
> >>> >> To unsubscribe, e-mail: users-
> >>> unsubscribe at gridengine.sunsource.net
> >>> >> For additional commands, e-mail: users-
> >>> help at gridengine.sunsource.net
> >>> >>
> >>> >>
> >>> >
> >>> >
> >>> --------------------------------------------------------------------
> >>> -
> >>> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>> > For additional commands, e-mail: users-
> >>> help at gridengine.sunsource.net
> >>> >
> >>>
> >>>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list