[GE users] Getting SGE to equally share resources

templedf dan.templeton at sun.com
Mon Feb 1 15:17:30 GMT 2010


Subordinate queues won't get you to even shares, but with that specific 
problem description, fair share + subordination might be a workable 
solution.  You'd have to move the 1-hour jobs into a separate queue, and 
make the other queue(s) subordinate to the 1-hour queue.  The fair-share 
scheduler does its fair sharing by looking at CPU consumption (at least 
by default), so the suspended jobs in the subordinate queue(s) won't 
count against the owners.

A less glamorous solution would be to separate the queues, but don't use 
subordination.  Instead use the queue's priority setting to turn up the 
nice value on the long-running jobs (or turn down the nice value on the 
1-hour jobs).  Basically, punt the problem down to the OS level.  It 
would get basically the same effect with less pomp and circumstance.

Daniel

On 02/01/10 06:39, scu98rkr wrote:
> Hi,
>
> It seems to me one of the failing of SGE is it still can not
> automatically evenly share resources between a group of users while
> allowing maximal use of the nodes for a single user if the space is
> available.
>
> I believe there was some discussion of this in the past and I believe
> something like this can be achieved by using subordinate queues.
>
>
> Basically say you have a cluster with 15 processor and 3 users. 1 user
> runs jobs that take 1 week, another 1 day and 1 user running jobs which
> last 1 hour and often the jobs relate to previous jobs.
>
> It is unlikely the user running jobs for 1 hours will be able to
> constantly keep the cluster flooded with jobs particularly over night or
> at the weekend.
>
> Therefore it would be best if the other two users could user this
> processor power. However these 2 users unlikely to have problems keeping
> the cluster flooded with jobs.
>
> If the queue is full of long running jobs the third users jobs
> automatically gets moved to the front of the queue but they may have to
> wait a day or even a week for their job to run.
>
> Ideally what should happen is, as well as the queue being spilt into
> thirds. The running jobs should be split into thirds. Therefore if any
> user is using more than a third of the cluster their jobs will be
> suspended and another users jobs ran. Therefore all users can have
> immediate access to the cluster if needed but also the processing power
> of the cluster is maximised.
>
> Is this possible using subordinate queues ? If not it really needs to be
> implemented as soon as is possible.
>
> Thanks Roger
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=242380
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=242393

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list