[GE users] Moved to users list: Re: [GE dev] Reservation

aeszter Ansgar.Esztermann at mpi-bpc.mpg.de
Thu May 20 09:07:32 BST 2010


    [ The following text is in the "utf-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

[Moved since I guess the users list is more appropriate now]

Hello everyone,

I would like some configuration advice. We have multiple queues per node, so we need some mechanism to prevent them from overloading a given node (e.g., an 8-core node should accept exactly eight processes, not eight from the "short" queue and another eight from "long"). In the past, we've used a "slots" complex on the nodes and an appropriate load_formula in sconf. However, I've just learned that putting queues into an alarm state is The Wrong Thing, and it will prevent reservations from working (see below).

Is there a better way to ensure a processes <= CPU cores relationship?


Thanks,

A.


On May 20, 2010, at 8:20 , andy wrote:
> 
> Hi,
> 
>> in our installation, reservations (the -R y kind, not advance
>> reservations) work, at best, intermittently. Therefore, I've generated
>> some debug traces and looked through the code (V62u5_TAG). However, there
>> is one point that is not quite clear to me:
>> 
>> in schedd/scheduler/dispatch_jobs(), the list of queues is filtered in
>> several steps. At one point, sge_split_queue_load(..., QU_load_thresholds)
>> removes overloaded queues. As far as I can see, these are never re-added,
>> so everything dependent on the queue list will have to work with the
>> reduced list of non-overloaded queues. In
>> sge_select_queue/parallel_tag_queues_suitable4job(), this queue list is
>> used to count the number of available slots at some point in the future in
>> order to make a reservation. Thus, it seems that queues overloaded *now*
>> will be ignored when considering a situation in the future. What am I
>> overlooking here?
> 
> The behavior you observe and what you've found out in the code is correct -
> at least that's how it's designed and intended to work. The motivation is
> that it's not known when an alarm state goes away - so it's not much
> different from a queue being in "unknown" state (I think the Advance
> Reservation does not select queues in alarm state as well). This behavior
> becomes problematic when you configured your SGE cluster that alarm states
> are "normal" when the queues are fully busy. In SGE's philosophy an alarm
> state is an exceptional, non-normal state - the goal should be to define
> queue slots as upper limits and when all queues slots are used and jobs are
> busy queues are not in alarm state.
> 
> I fully understand there can be quite different views on this behavior:-)
> 
> Changing the scheduler code would be likely non-trivial since as you see
> from the code reservations are done during a normal scheduling run when non
> reservation jobs are dispatched as well - of course you don't want jobs are
> dispatched to queues which are in alarm state.
> 
> Andy
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=39&dsMessageId=257940
> 
> To unsubscribe from this discussion, e-mail: [dev-unsubscribe at gridengine.sunsource.net].


-- 
Ansgar Esztermann
DV-Systemadministration
Max-Planck-Institut für biophysikalische Chemie, Abteilung 105

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=257951

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list