[GE users] Per-node group priorities?
dag at sonsorol.org
Thu Dec 11 00:10:32 GMT 2008
On Dec 10, 2008, at 5:20 PM, Jim Zajkowski wrote:
> On 12/10/08 5:14 PM, "craffi" <dag at sonsorol.org> wrote:
>> On Dec 10, 2008, at 3:12 PM, Jim Zajkowski wrote:
>>> Is it possible to configure SGE so that jobs submitted by members of
>>> a group have a preference for a specific set of nodes?
It is possible to to do this but it sounds like you may be more
interested in letting the 2 PIs use any server in the cluster as long
as the total allocation mix is 50-50 when there is activity from both
>> - Do you have a specific set of 20 machines you want to use for these
>> 2 groups or do you want to pull any 20 nodes out of a much larger
> I am indifferent - we simply ordered 20 identical machines (Sun Blade
> x6250), and they're all racked together and share the same SAN, etc.
>> - Would any other users or groups active on these 20 servers?
Then it sounds like a simple Department or Project based functional or
sharetree policy with a simple 50% entitlement spread among users from
the two labs would work fine. Resource quotas can further constrain
things or can be used as an emergency threshold to prevent one PI from
grabbing 100% of all available slots on an idle cluster.
The big thing to note (as I think you already picked up) is that SGE
by default is not going to mess with running jobs. All SGE policies
are enforced by the SGE scheduler constantly adjusting the jobs
waiting in the pending list. This means that one of your PIs could
swamp the cluster for a bit (getting a 100% mix) and you would go down
to the desired 50-50 mix as active jobs drain out. Since you mentioned
having fast running jobs this is likely not going to be a big deal for
I wrote up a blog post covering the difference between Share Tree and
Functional policies. The main difference is if you want your 50-50 mix
to be calculated via the current cluster usage (functional policy) or
if you want the scheduler to look back over a period of time to
account for prior usage when doing calculations (share tree)
The url is here:
My experience suggest that while Share Trees are attractive to an
administrator and operator they tend to be viewed suspiciously by end-
users. When your PI comes to you demanding to know why they are not
getting their 10 node allocation it's hard to go back and "prove" to
them that the scheduler is just adjusting a little bit for past usage
in which they benefited from much more than their official allocation.
>>> the other being that the tools being run already expect an SGE
>> Out of curiosity, what are the tools?
> One of the PIs has brought along the scripts and procedures from his
> previous location. They are biologists and not computer scientists,
> prefer little disruption as possible. That being said I'm open to
Clustering for Biology/life-science is my primary area of
concentration as well. SGE is quite popular with our crowd.
To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
More information about the gridengine-users