[GE users] High and low priority queues, no starvation, minimum oversubscription

Reuti reuti at staff.uni-marburg.de
Sat Mar 4 02:01:33 GMT 2006


Hi Ian,

Am 04.03.2006 um 01:02 schrieb Ian Fasel:

> I'm trying to make a design where everyone gets a preferred high  
> priority machine, but then shares equally for the remaining  
> machines.   I also want to prevent jobs from getting completely  
> suspended by higher-priority jobs.
>
> I have almost done this.  Each user has 2 queues, a high-prio (HP)  
> and a low-prio (LP)  queue.  Each queue has two slots per host.  On  
> the HP queue, you get your preferred host.  It has sequence number  
> 10.  Your LP queue includes all the remaining hosts, and it is set  
> to niceness 20, and sequence number 50.  Finally, there is a  
> total_slots consumable resource, and each host is configured to  
> have 2 total_slots, and each job requests one of these by default.   
> Scheduling is done using sequence number.  We also have share  
> tickets so scheduling of jobs is fair.
>
> All would be good except for these two things:  How can I get HP  
> jobs to not *require* the total_slots to be > 0 in order to start  
> running, however I do want jobs in HP queues to actually decrease  
> total_slots resource so that others can't use it until it is > 0?   
> Probably total_slots should actually be total_lowprio_slots.
>

I think it could be done with a consumable complex as load threshold.  
Please have a look at:

http://gridengine.sunsource.net/issues/show_bug.cgi?id=464

which works now (my free_slots could be your total_slots). The  
"common" queue could be the one for the secondary user, and the  
"urgent" one for the (high priority) main user of this machine.

HTH - Reuti


> Here's a scenario and the desired result:
>
> 1) User1 sends four jobs.  He gets two on his preferred machine,  
> one on each of the others.
>
> slot:  job owner:
> A.1   User1
> A.2   User1
> B.1   User1
> B.2   --
> C.1   User1
> C.2   --
>
> 2) User2 sends four jobs. Immediately he gains two jobs on hostB,  
> he is unable to get any on machine 1 because UserA is using it, and  
> he gets one on hostC.  One job is waiting.
>
> slot:  job owner:
> A.1   User1
> A.2   User1
> B.1   User2, User1 (nice 20)
> B.2   User2
> C.1   User1
> C.2   User2
>  --1 queued waiting from User2
>
> 3) Finally one job in the high priority queue from User 1 finishes,  
> and the nice job from User1 also finishes:
>
> A.1   User1
> A.2   User2
> B.1   User2,
> B.2   User2
> C.1   User1
> C.2   User2
>
> Thanks for your help!
> -Ian
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list