[GE users] Allocating CPUs team wise

Chris Dagdigian dag at sonsorol.org
Wed Mar 16 11:07:28 GMT 2005

Hi Nitin,

SGE 6 has excellent mechanisms for doing resource allocation, especially 
when it comes to groups of users, departments or projects.

There are a number of ways you can get what you are requesting - for 
instance if you really wanted to be strict about which CPUs each of your 
  team members uses, you could take advantage of then Access Control 
Lists and XACLs that Grid Engine offers. This would involve:

o Configuring SGE to be aware of each user and making a usergroup object 
for each "team". Then you could use SGE's ACL and XACL mechanisms to 
control at the queue level who is allowed to run on each queue instance. 
You could also do this with SGE Department or SGE Project objects.

All this is well covered in the SGE documentation collection:

This would get you strictly what you want but is not extremely flexible 
and would require effort on your part to constantly adjust and move 
around the ACLs to reflect current needs.

A more "flexible" approach may be to use the SGE Sharetree or Functional 
Share policies to do roughly the same thing. Using functional shares it 
is easy to set up policies that do something like this:

  o when the cluster is idle; any team can run on any CPU
  o when cluster is busy; Team A gets 30% of cluster resources
  o when cluster is busy; Team B gets 70% of cluster resources

The sharetree or functional policy may allow you to allocate cluster 
resources in a way that is more efficient and easier for you to manage 
on a day-to-day basis. It is certainly less "work" for an admin than 
having to constantly manipulate access control lists.

The hard part about the functional/share-tree policy is actually 
explaining the system to end users, management and "stakeholders". In my 
experience people always want constant assurance that they are not being 
"cheated" out of their rightful share of compute power. ACLs and how 
they behave are easily monitored by end users; the resource allocation 
policies are a bit more hidden as they operate within the scheduler and 
work only to adjust the priority of jobs sitting in the pending list.

It's not totally appropriate to your situation but in the past when I 
had to divide a cluster up for usage among different departments I took 
careful notes and threw it up on the web. There may be useful 
information there: http://bioteam.net/dag/sge6-funct-share-dept.html


Nitin Raina wrote:
> Hello All,
> We are using SGE 6.0 for submitting serial and parallel jobs across our 
> cluster. This cluster is used by various teams .We have two queues 
> created which has set of users bound with it who can run jobs on the 
> respective queues.
> Further to this arrangement we are thinking of a way by which we can 
> merge both the queues and in return create a arrangement where we can 
> allot CPUs team wise.
> We would like to have let say 10 CPUs dedicated to Team A, 8 CPUS 
> dedicated to Team B and so on..
> Is that possible? If yes then can we have this arrangement in a dynamic 
> way by which I can reduce and increase the count of CPUs as per the need 
> arises.
> Any help would be much appreciated.

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list