[GE users] Overlapping hostgroups in cluster queues

Craig Tierney ctierney at hpti.com
Mon Sep 6 17:32:57 BST 2004

I am installing SGE 6.0 on a system with cluster queues
and parallel environments.  In the end I want to be able
to specify the -masterq for some of the jobs.  The nodes
that could be specified on the masterq line are subset of
the total nodes and those nodes should never be scheduled
in general.  You might want to do this if a couple of nodes
in your cluster have more memory or a faster connection to storage
for the rank 0 process in an mpi job.

So I create 2 hostgroups, @comp and @io.  The hostgroup
@comp is g0001-g0030.  The hostgroup @io is "g0000 @comp".
Then I create two cqueues and two parallel environments based
on those hostgroups.  SLOTS is set to two for each cqueue
(each node is 2 processor).  The allocation rule is $fill_up
and each job asks for an even number of processors to ensure
only one job gets scheduled to a node at a time.

The problem is that hosts do not have a SLOTS value, only queues.
So now I have qcomp1.q at g0005 and qio1.q at g0005.  Each is defined
to have 2 slots, but each definition is separate so that
if I submit 1 job to pecomp1 and peio1 I can get 4 processes
running on host g0005.  There should only be 2 processes running.

I think I get get the right behavior if I throw out the cqueues
and hostgroups,  define a queue for each host and then specify
both pecomp1 and peio1 for each queue that I created.  But that
is a bit messy and doesn't allow me to use cluster queues which
are supposed to simplify this process.

Do I need to just drop back to not using cluster queues?
Anyone have an idea of how I can do this?  


To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list