[GE users] Allowing parallel jobs to run on busy cluster

Sean Davis sdavis2 at mail.nih.gov
Wed Dec 3 23:41:37 GMT 2008


On Wed, Dec 3, 2008 at 5:34 PM, craffi <dag at sonsorol.org> wrote:
> Hi Sean,
>
> "parallel job starvation" used to be a problem on older SGE systems as
> faster non-parallel jobs would zip in and out of job slots during each
> scheduling interval while never leaving enough free slots for the
> larger parallel job.
>
> I think this has been mostly solved now in recent versions of SGE via
> the use of reservation based scheduling. People submit the parallel
> job with a reservation request (-R y ) and SGE will slowly start to
> hold back job slots until there are enough free.
>
> Hopefully someone will correct me ASAP if I got the above bit wrong.
> Back in the day I recall dealing with this issue via the
> wait_wait_time Urgency sub-policy but I don't think that hack is
> needed anymore.

That does it, I think.  The following link has a fair bit of
information on the topic in case anyone wants more detail.

http://gridengine.sunsource.net/nonav/source/browse/~checkout~/gridengine/doc/devel/rfe/resource_reservation.txt?content-type=text/plain

> On Dec 3, 2008, at 5:23 PM, Sean Davis wrote:
>
>> We have a small cluster that consists of several SMP machines.  We
>> have been running into the situation that many serial jobs have been
>> submitted (and are in the queue) and a user wants to run an 8-process
>> parallel process on a single SMP machine.  Is it possible without
>> setting arbitrary resource limits on particular users (those
>> submitting the serial jobs) to give priority to the parallel job?
>> Otherwise, what happens is that the serial jobs always take precedence
>> until there are enough processors open on a single machine to run the
>> parallel job (the PE is set up as $pe_slots).
>>
>> Thanks,
>> Sean
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=90968
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=90981

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list