[GE users] High and low priority queues, no starvation, minimum oversubscription

Ian Fasel ianfasel at mplab.ucsd.edu
Sat Mar 4 00:02:11 GMT 2006

I'm trying to make a design where everyone gets a preferred high  
priority machine, but then shares equally for the remaining  
machines.   I also want to prevent jobs from getting completely  
suspended by higher-priority jobs.

I have almost done this.  Each user has 2 queues, a high-prio (HP)  
and a low-prio (LP)  queue.  Each queue has two slots per host.  On  
the HP queue, you get your preferred host.  It has sequence number  
10.  Your LP queue includes all the remaining hosts, and it is set to  
niceness 20, and sequence number 50.  Finally, there is a total_slots  
consumable resource, and each host is configured to have 2  
total_slots, and each job requests one of these by default.   
Scheduling is done using sequence number.  We also have share tickets  
so scheduling of jobs is fair.

All would be good except for these two things:  How can I get HP jobs  
to not *require* the total_slots to be > 0 in order to start running,  
however I do want jobs in HP queues to actually decrease total_slots  
resource so that others can't use it until it is > 0?  Probably  
total_slots should actually be total_lowprio_slots.

Here's a scenario and the desired result:

1) User1 sends four jobs.  He gets two on his preferred machine, one  
on each of the others.

slot:  job owner:
A.1   User1
A.2   User1
B.1   User1
B.2   --
C.1   User1
C.2   --

2) User2 sends four jobs. Immediately he gains two jobs on hostB, he  
is unable to get any on machine 1 because UserA is using it, and he  
gets one on hostC.  One job is waiting.

slot:  job owner:
A.1   User1
A.2   User1
B.1   User2, User1 (nice 20)
B.2   User2
C.1   User1
C.2   User2
  --1 queued waiting from User2

3) Finally one job in the high priority queue from User 1 finishes,  
and the nice job from User1 also finishes:

A.1   User1
A.2   User2
B.1   User2,
B.2   User2
C.1   User1
C.2   User2

Thanks for your help!

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list