[GE users] Multiple users and policies

Chris Dagdigian dag at sonsorol.org
Tue Sep 16 19:55:22 BST 2008


Sean,

By default (as you know) the Grid Engine scheduler will dispatch job  
first-come-first-serve in the default configuration.

To get away from this behavior you have to activate one of the SGE  
resource allocation policies.

The easiest and simplest method is to choose the "Functional" policy  
and set that policy up such that it considers all users as equal.

The end result of that is "fairshare by user" where users are treated  
fairly. In this context you can submit jobs in any order and SGE will  
start to inter-mingle them to achieve the fairest possible mix.

The one thing to understand is that SGE implements policies by  
manipulating and constantly re-ordering the list of jobs that are  
pending dispatch. It *DOES NOT* mess with running jobs unless you go  
out of your way to configure such actions.

What this means is that your cluster will be "fair" but it will only  
be able to be "fair" by filling in slots as running jobs drain out and  
exit from the system. If your first user comes in on Monday morning  
and submits three-week jobs that consume all available job slots then  
the SGE scheduler is not going to be able to allocate "fairly" until  
some of those first jobs are killed or drain out naturally.

There are other mechanisms (policy tweaks, max_u_jobs, resource  
quotas) to deal with "one user swamping the entire cluster" so I'd  
stat first with experimenting with the fairshare by user policy.

Fairshare by user via the Functional policy can be set up in a few  
seconds using just 2 or three SGE qconf commands.

It's written up here:
http://gridengine.info/2006/01/17/easy-setup-of-equal-user-fairshare-policy


Regards,
Chris


On Sep 16, 2008, at 2:40 PM, Sean Davis wrote:

> Up until now, our group has delegated all SGE job submission tasks to
> a single person (people do not like to learn new tricks).  Now, we are
> going to have multiple users submitting jobs.  How can I ask SGE to
> allow users' jobs to intermingle?  To give a concrete example, say I
> submit 100 jobs and then a colleague submits 100 jobs.  I would like
> my colleague's jobs to begin running immediately.  How can I tell the
> scheduler to trickle jobs from various users on not exactly a
> first-in-first-out model?
>
> Thanks,
> Sean
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list