[GE users] High and low priority queues, no starvation, minimum oversubscription

Ian Fasel ianfasel at mplab.ucsd.edu
Sat Mar 4 22:48:41 GMT 2006


Yes, I think that will work.  Thanks!

On Mar 3, 2006, at 6:01 PM, Reuti wrote:

> Hi Ian,
>
> Am 04.03.2006 um 01:02 schrieb Ian Fasel:
>
>> I'm trying to make a design where everyone gets a preferred high  
>> priority machine, but then shares equally for the remaining  
>> machines.   I also want to prevent jobs from getting completely  
>> suspended by higher-priority jobs.
>>
>> I have almost done this.  Each user has 2 queues, a high-prio (HP)  
>> and a low-prio (LP)  queue.  Each queue has two slots per host.   
>> On the HP queue, you get your preferred host.  It has sequence  
>> number 10.  Your LP queue includes all the remaining hosts, and it  
>> is set to niceness 20, and sequence number 50.  Finally, there is  
>> a total_slots consumable resource, and each host is configured to  
>> have 2 total_slots, and each job requests one of these by  
>> default.  Scheduling is done using sequence number.  We also have  
>> share tickets so scheduling of jobs is fair.
>>
>> All would be good except for these two things:  How can I get HP  
>> jobs to not *require* the total_slots to be > 0 in order to start  
>> running, however I do want jobs in HP queues to actually decrease  
>> total_slots resource so that others can't use it until it is > 0?   
>> Probably total_slots should actually be total_lowprio_slots.
>>
>
> I think it could be done with a consumable complex as load  
> threshold. Please have a look at:
>
> http://gridengine.sunsource.net/issues/show_bug.cgi?id=464
>
> which works now (my free_slots could be your total_slots). The  
> "common" queue could be the one for the secondary user, and the  
> "urgent" one for the (high priority) main user of this machine.
>
> HTH - Reuti
>
>
>> Here's a scenario and the desired result:
>>
>> 1) User1 sends four jobs.  He gets two on his preferred machine,  
>> one on each of the others.
>>
>> slot:  job owner:
>> A.1   User1
>> A.2   User1
>> B.1   User1
>> B.2   --
>> C.1   User1
>> C.2   --
>>
>> 2) User2 sends four jobs. Immediately he gains two jobs on hostB,  
>> he is unable to get any on machine 1 because UserA is using it,  
>> and he gets one on hostC.  One job is waiting.
>>
>> slot:  job owner:
>> A.1   User1
>> A.2   User1
>> B.1   User2, User1 (nice 20)
>> B.2   User2
>> C.1   User1
>> C.2   User2
>>  --1 queued waiting from User2
>>
>> 3) Finally one job in the high priority queue from User 1  
>> finishes, and the nice job from User1 also finishes:
>>
>> A.1   User1
>> A.2   User2
>> B.1   User2,
>> B.2   User2
>> C.1   User1
>> C.2   User2
>>
>> Thanks for your help!
>> -Ian
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list