[GE users] job scheduling on different queues

jcoomer james.coomer at sun.com
Fri Jul 31 11:13:10 BST 2009


Hi Mat,

I thought there might be a complication there. By the way, I remember 
that you have subordinating queues to ensure exclusive host access for 
parallel jobs - and that can now be achieved in other ways with the 
latest grid engine version which might be tidier.

Thomas - perhaps there is a better way of doing things without mutually 
subordinating queues. Have you implemented the queues purely to create 
this prioritisation? Obviously it isn't generally necessary to have 
different queues for serial and parallel jobs.

James


matbradford wrote:
> James,
>
> We had the same problem as Thomas. With mutually subordinating queues,
> resource reservation doesn't work. Or didn't in 6.1. I think the problem
> is that as the queues are suspended, they don't get included in the list
> of available queues for reservation.
>
> I think Resource reservation only works when there is a single, or at
> least dominant, queue on the nodes.
>
> Cheers,
>
> Mat
>
>   
>> -----Original Message-----
>> From: James.Coomer at sun.com [mailto:James.Coomer at sun.com]
>> Sent: 30 July 2009 19:42
>> To: users at gridengine.sunsource.net
>> Subject: Re: [GE users] job scheduling on different queues
>>
>> Hi Thomas,
>>
>> This is called "job starvation" and gridengine has resource reservation
>> available to counter the problem. I'm not sure if, because you have
>> multiple queues, things are behaving differently  - but first, try
>> switching on "resource reservation"
>>
>> here's some information  -but now you know the feature name, you should
>> find a load of info.
>>
>> http://gridengine.info/2006/05/31/resource-reservation-prevents-
>> parallel-job-starvation
>>
>> James
>>
>> tgebert wrote:
>>     
>>> Hello list,
>>>
>>> I have node1 - node4 (four CPUs on each node) on my cluster and
>>>       
>> configured two queues for parallel jobs and one queue for serial jobs
>> (q_par and q_ser).
>>     
>>> Both queues can access all cores on all nodes so every queue has 16
>>>       
>> slots. The queues are configured to suspend the other queue through the
>> Subordinate feature.
>>     
>>> The probelm I am facing is that if there are e.g 3 jobs running in
>>>       
> the
>   
>> q_ser and there is a parallel job waiting in the queue q_par, later
>> submitted serial jobs to q_ser are started immediately even if the
>> parallel job has a higher priority. This would lead to that the
>>     
> parallel
>   
>> job is never started in the worst case.
>>     
>>> Does anyone know if it's possible to configure the scheduling in a
>>>       
>> way, that  the newly submitted serial jobs are not started immediately
>> and wait until the resources for the parallel job are freed, so the
>> parallel job can start?
>>     
>>> All the Best
>>> Thomas
>>>
>>> ------------------------------------------------------
>>>
>>>       
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessag
>>     
> e
>   
>> Id=210263
>>     
>>> To unsubscribe from this discussion, e-mail: [users-
>>>       
>> unsubscribe at gridengine.sunsource.net].
>>     
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessag
>>     
> e
>   
>> Id=210290
>>
>> To unsubscribe from this discussion, e-mail: [users-
>> unsubscribe at gridengine.sunsource.net].
>>     
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=210394
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=210396

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list