[GE users] Scheduling based on queues having the greatest/least value of a resource?

Mike Cumings mcumings at gmail.com
Wed Sep 10 03:45:19 BST 2008


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Reuti (and others that responded off the list),

Thanks for the responses and ideas.  I'll definitely need to look into
the sequence numbers to see if that will suffice as a solution.  I may
be able to simply redefine my problem to make it fit the sequence
number idea.

To provide a bit more context - which I intentionally left out in my
first exchange - we're using SGE in a very odd manner.  We're using it
as a scheduling solution for lab machines.  Specifically, for
temporarily assigning lab machines to test cases which test software
installations against bare metal.  Our use cases defines the software
load servers as our executions hosts (provided implicit references to
required installation infrastructure) and a queue to represent each
lab system.  Software loading "jobs" get dispatched to available lab
system "queues" for software installation testing.  In this case, the
resource I referred to is the documented average software installation
speed (or time).   For tests which require full software installations
to run, we desire systems which load the software at the fastest rate
to be selected first.  For tests which run in "pretend" mode (i.e.,
test other parts of the software system without requiring an actual
software installation) the load time is irrelevant so we'd prefer to
consume the slowest loading lab machines first such that we leave the
other lab machines free for the actual software loads. Thus, being
able to invert the scheduling criteria per job is desired.  From this
description, you can also see that as newer/faster machines are added
to the grid over time that we'd like to have these systems naturally
preferred over the previous generation of slower systems.  There are
also additional complications that I'm not mentioning here (e.g.,
selection of logical domains of the hardware system), but this is the
gist of our use case.

Admittedly, this is a complete bastardization of what SGE was built to
solve, but we've found it to be extremely useful in this role.  ;)

Mike

On Mon, Sep 8, 2008 at 3:17 AM, Reuti <reuti at staff.uni-marburg.de> wrote:
> Hi,
>
> Am 05.09.2008 um 18:51 schrieb Mike Cumings:
>
>> I have a grid with many queues per system, each queue having an
>> integer resource defined on it.  Due to the way our application works,
>> we expect more queues with greater values of this resource value to be
>
> what does this recource represent? Diskspace, memory, ...?
>
>> added over time.  It is expected that over time we will gradually
>> retire queues with the lowest resource values to make room for new
>
> What is the reason for this setup, I mean: when you get e.g. more memory in
> some machines, you could just change the value of it for this machine in
> SGE. There is no need to setup a special queue for it.
>
> The usual approach might be to setup "seq_no" for these queues (in the queue
> definition), and change the scheduler behavior to "queue_sort_method
>         load". But even with just one queue, you could give different
> sequence numbers to different hosts.
>
> -- Reuti
>
>
>> queues with higher resource values.  With this context in mind, we'd
>> like to be able to have jobs be assigned to these queues such that
>> queues with higher values for this numeric resource are favored over
>> queues with lower values.  Ideally, we'd also like to be able to
>> invert this comparison such that jobs could be submitted and favor
>> queues with the lowest numeric values for this resource.  Is this
>> possible?
>>
>> --
>> Mike Cumings
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>



-- 
Mike Cumings

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list