[GE users] Allocating CPUs team wise
dag at sonsorol.org
Wed Mar 16 11:07:28 GMT 2005
SGE 6 has excellent mechanisms for doing resource allocation, especially
when it comes to groups of users, departments or projects.
There are a number of ways you can get what you are requesting - for
instance if you really wanted to be strict about which CPUs each of your
team members uses, you could take advantage of then Access Control
Lists and XACLs that Grid Engine offers. This would involve:
o Configuring SGE to be aware of each user and making a usergroup object
for each "team". Then you could use SGE's ACL and XACL mechanisms to
control at the queue level who is allowed to run on each queue instance.
You could also do this with SGE Department or SGE Project objects.
All this is well covered in the SGE documentation collection:
This would get you strictly what you want but is not extremely flexible
and would require effort on your part to constantly adjust and move
around the ACLs to reflect current needs.
A more "flexible" approach may be to use the SGE Sharetree or Functional
Share policies to do roughly the same thing. Using functional shares it
is easy to set up policies that do something like this:
o when the cluster is idle; any team can run on any CPU
o when cluster is busy; Team A gets 30% of cluster resources
o when cluster is busy; Team B gets 70% of cluster resources
The sharetree or functional policy may allow you to allocate cluster
resources in a way that is more efficient and easier for you to manage
on a day-to-day basis. It is certainly less "work" for an admin than
having to constantly manipulate access control lists.
The hard part about the functional/share-tree policy is actually
explaining the system to end users, management and "stakeholders". In my
experience people always want constant assurance that they are not being
"cheated" out of their rightful share of compute power. ACLs and how
they behave are easily monitored by end users; the resource allocation
policies are a bit more hidden as they operate within the scheduler and
work only to adjust the priority of jobs sitting in the pending list.
It's not totally appropriate to your situation but in the past when I
had to divide a cluster up for usage among different departments I took
careful notes and threw it up on the web. There may be useful
information there: http://bioteam.net/dag/sge6-funct-share-dept.html
Nitin Raina wrote:
> Hello All,
> We are using SGE 6.0 for submitting serial and parallel jobs across our
> cluster. This cluster is used by various teams .We have two queues
> created which has set of users bound with it who can run jobs on the
> respective queues.
> Further to this arrangement we are thinking of a way by which we can
> merge both the queues and in return create a arrangement where we can
> allot CPUs team wise.
> We would like to have let say 10 CPUs dedicated to Team A, 8 CPUS
> dedicated to Team B and so on..
> Is that possible? If yes then can we have this arrangement in a dynamic
> way by which I can reduce and increase the count of CPUs as per the need
> Any help would be much appreciated.
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users