[GE users] job scheduling on different queues

matbradford matthew.bradford at eds.com
Fri Jul 31 15:08:23 BST 2009


James,

We are looking at exclusive scheduling, but currently it's a big change
to our config, so are being somewhat cautious.

We may wait a while to see whether any issues come out of the woodwork.

Cheers,

Mat

>-----Original Message-----
>From: James.Coomer at sun.com [mailto:James.Coomer at sun.com]
>Sent: 31 July 2009 11:13
>To: users at gridengine.sunsource.net
>Subject: Re: [GE users] job scheduling on different queues
>
>Hi Mat,
>
>I thought there might be a complication there. By the way, I remember
>that you have subordinating queues to ensure exclusive host access for
>parallel jobs - and that can now be achieved in other ways with the
>latest grid engine version which might be tidier.
>
>Thomas - perhaps there is a better way of doing things without mutually
>subordinating queues. Have you implemented the queues purely to create
>this prioritisation? Obviously it isn't generally necessary to have
>different queues for serial and parallel jobs.
>
>James
>
>
>matbradford wrote:
>> James,
>>
>> We had the same problem as Thomas. With mutually subordinating
queues,
>> resource reservation doesn't work. Or didn't in 6.1. I think the
>problem
>> is that as the queues are suspended, they don't get included in the
>list
>> of available queues for reservation.
>>
>> I think Resource reservation only works when there is a single, or at
>> least dominant, queue on the nodes.
>>
>> Cheers,
>>
>> Mat
>>
>>
>>> -----Original Message-----
>>> From: James.Coomer at sun.com [mailto:James.Coomer at sun.com]
>>> Sent: 30 July 2009 19:42
>>> To: users at gridengine.sunsource.net
>>> Subject: Re: [GE users] job scheduling on different queues
>>>
>>> Hi Thomas,
>>>
>>> This is called "job starvation" and gridengine has resource
>reservation
>>> available to counter the problem. I'm not sure if, because you have
>>> multiple queues, things are behaving differently  - but first, try
>>> switching on "resource reservation"
>>>
>>> here's some information  -but now you know the feature name, you
>should
>>> find a load of info.
>>>
>>> http://gridengine.info/2006/05/31/resource-reservation-prevents-
>>> parallel-job-starvation
>>>
>>> James
>>>
>>> tgebert wrote:
>>>
>>>> Hello list,
>>>>
>>>> I have node1 - node4 (four CPUs on each node) on my cluster and
>>>>
>>> configured two queues for parallel jobs and one queue for serial
jobs
>>> (q_par and q_ser).
>>>
>>>> Both queues can access all cores on all nodes so every queue has 16
>>>>
>>> slots. The queues are configured to suspend the other queue through
>the
>>> Subordinate feature.
>>>
>>>> The probelm I am facing is that if there are e.g 3 jobs running in
>>>>
>> the
>>
>>> q_ser and there is a parallel job waiting in the queue q_par, later
>>> submitted serial jobs to q_ser are started immediately even if the
>>> parallel job has a higher priority. This would lead to that the
>>>
>> parallel
>>
>>> job is never started in the worst case.
>>>
>>>> Does anyone know if it's possible to configure the scheduling in a
>>>>
>>> way, that  the newly submitted serial jobs are not started
>immediately
>>> and wait until the resources for the parallel job are freed, so the
>>> parallel job can start?
>>>
>>>> All the Best
>>>> Thomas
>>>>
>>>> ------------------------------------------------------
>>>>
>>>>
>>>
>http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessag
>>>
>> e
>>
>>> Id=210263
>>>
>>>> To unsubscribe from this discussion, e-mail: [users-
>>>>
>>> unsubscribe at gridengine.sunsource.net].
>>>
>>> ------------------------------------------------------
>>>
>http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessag
>>>
>> e
>>
>>> Id=210290
>>>
>>> To unsubscribe from this discussion, e-mail: [users-
>>> unsubscribe at gridengine.sunsource.net].
>>>
>>
>> ------------------------------------------------------
>>
>http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessag
e
>Id=210394
>>
>> To unsubscribe from this discussion, e-mail: [users-
>unsubscribe at gridengine.sunsource.net].
>>
>
>------------------------------------------------------
>http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessag
e
>Id=210396
>
>To unsubscribe from this discussion, e-mail: [users-
>unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=210435

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list