[GE users] Questions on functional policy, share tree policy, and priority queue

Steve Chapel steve.chapel at a2pg.com
Wed May 21 01:29:21 BST 2008


I'm new to Sun Grid Engine. I have it set up on a small Rocks cluster
running SGE 6.0 for the three owners of the company I work for, but want a
more fair scheduling policy than the default FIFO policy. I want to ensure
that all three owners get approximately equal access to the cluster. All
jobs are single batch jobs or array batch jobs, in an embarrassingly
parallel situation. Each of the compute nodes has two quad-core processors.

In particular, I want to ensure the following:
1. If there is no contention for CPUs, a user should get access to as many
CPUs as they need (equal to either the number of jobs they have run or the
number of CPUs, whichever is less).
2. If two or more users need to compete for CPUs, they should each get a
"fair share" of CPU time.
3. In case of urgent need, we should be able to run jobs that will preempt
any currently running jobs.

It seems to me that I want to set up either a functional policy or a share
tree policy. In addition, I want a queue that supersedes the default all.q.
I have read through the documentation, and it looks like this is what I
should set up:
* Add high-priority queue: qconf -aq priority.q
* Add all.q to priority.q subordinate list: qconf -mattr queue
subordinate_list all.q priority.q
* Set priority of priority.q to highest: qconf -mattr queue priority -20
* In conf, set auto_user_fshare to 100
* Check that fshare is 100 for all users listed in qconf -suserl
* In sconf, set halftime to 48, compensation_factor to 5
* Add stree: id=0, name=default, type=0, shares=100, childnodes=NONE
* For functional policy: in sconf, set weight_tickets_functional to 10000,
weight_tickets_share to 0
* For share tree policy: in sconf, set weight_tickets_functional to 0,
weight_tickets_share to 10000

Some specific questions I have on these settings are:
1. Is there a possibility that a user may not be able to use all available
CPUs on the cluster if there are enough jobs to use all CPUs?
2. Will the functional and share tree policy settings not interfere with
each other as long as either weight_tickets_functional or
weight_tickets_share are 0?
3. Will submitting jobs to priority.q always immediately cause at least some
currently running jobs to be suspended? Will it cause all currently running
jobs to be suspended if a user submits enough jobs?
4. Does setting the -20 priority of priority.q have an effect?
5. Are there any settings I have missed?


To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list