[GE users] Scheduler Configuration

Robert Healey healer at rpi.edu
Tue Dec 23 06:10:14 GMT 2008


I've been running 6.2 for the past 3 months and am running up against a 
wall trying to configure the system for the per node performance that 
pen and paper scheduling gave my users. I currently have two physical 
clusters and 3 queues configured.

Cluster 1 is 4 racks of dual cpu opterons and Cluster 2 is 4 racks of 8 
way xeons.  The first queue is configured for multiproc only jobs on all 
the opterons, the second for multiproc only jobs on the xeons, and the 
last queue is configured for single proc jobs on rack #4 of the opterons 
and the xeons.

In my PE configuration, if I set the allocation rule to $fill_up I get a 
very inefficient distribution a job that uses a full node's worth of 
cores gets distributed across at least two nodes.  This is less of an 
issue on the opterons than the xeons.  Its not an even split either, but 
5/3, 7/1, 6/2, etc.  very rarely 4/4.  If I set the allocation rule for 
the xeon PE to 8, on xeon rack #4 I end up with the 8 mpi threads for a 
PE job on a node and 8 single core jobs also on the same node, resulting 
in very poor performance.

Maybe I can't mix serial and parallel queues on the same node, but if I 
can, if anyone has some pointers on how to straighten this out without 
reverting back to using pen/paper to assign nodes to researchers, it 
would be appreciated.

Thank you very much.

qconf -msched:
algorithm                         default
schedule_interval                 0:0:15
maxujobs                          0
queue_sort_method                 seqno
job_load_adjustments              NONE
load_adjustment_decay_time        0:0:0
load_formula                      slots
schedd_job_info                   true

Bob Healey
Systems Administrator
Physics Department, RPI
healer at rpi.edu


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list