[GE users] Preemption vs dedicating nodes by group?

Justus Loerke loerke at molgen.mpg.de
Thu Apr 28 17:43:38 BST 2005

Hm, I think I may be looking at some similar setup to balance load 
between two user groups... I just configured primary/secondary queues 
like this: primary queues open only to specified user lists via user 
access in the queue configuration (giving some kind of dedicated 
queues), secondary queues open to all users, but I added checkpointing 
to kill & restart jobs on secondary queues instead of suspending or 
disabling them. Jobs are submitted without specification of the queue, I 
let GE worry about sending the jobs to the right nodes. (Note: I have 
one primary and secondary queue for every single machine, is it possible 
to configure multi-host queues with 5.3p6? ) Sec. queues are configured 
subordinated to primary, so once the primary queue is empty, jobs will 
be started on secondary queues, but will be terminated immediately (and 
eventually restarted) once a job in the dedicated primary queue is started.
The setup is something like this:
-configure primary and secondary queues, secondary subordinated.
-add userdefined checkpointing scheme, add the restart on suspend option 
and add all secondary queues for this checkpointing scheme
-add "-ckpt scheme" to the qsub command
This way I don't even need a load sensor to check queue contents (what I 
looked at first), like Reuti suggested previously. In effect, this kills 
secondary jobs once a primary one comes along to take the dedicated 
queue. This may be troublesome if you're worried about file corruption 
by killed jobs (like I am, since so far we haven't really been using 
checkpointing on any level at all), but maybe this can be solved on the 
scripting level (but I haven't looked into that yet).
I hope I'm making some sense, been a long day ;)



Dipl. Phys. Justus Loerke
- UltraStrukturNetzwerk -
Max Planck Institute for Molecular Genetics
Ihnestr. 63-73
D-14195 Berlin

Tel.:   +49-30-8413-1644
Fax:    +49-30-8413-1385
E-mail: loerke at molgen.mpg.de  

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list