[GE users] optimize job distribution for license usage?

reuti reuti at staff.uni-marburg.de
Fri Dec 4 19:52:44 GMT 2009


Hi,

Am 04.12.2009 um 17:44 schrieb ruppert:

> we want to use an application with GridEngine which is licensed
> according to "user at host": starting the application multiple times
> by the same user on the same host requires only one license token,
> whereas another user on the same host (or the same user on another
> host) requires additional license tokens.

interesting. This would imply that you get more from the license if  
you buy a single machine with eight installed quad-cores, instead of  
buying eight machines with two dual-cores installed each.

a) How many cores per machine you have right now?

b) How man cores the users usually request for their jobs?

c) How is the limit of the licenses checked? SGE needs to know  
beforehand already, whether the job can be scheduled at all.

First idea could be to use a fixed alloaction_rule in your PE setup,  
i.e. enter a plain number there like 4 or 2 to lower fragmentation.

-- Reuti


> Assume that there are two hosts A and B, each capable of running
> two jobs in parallel, i.e. four jobs could run at the same time.
> If two users each submit two jobs, this would require two license
> tokens if the jobs of one user both end up on host A, and the jobs
> of the other user on host B. Four license tokens would be needed
> if the two jobs of each user end up on different hosts.
>
> From this licensing scheme, it seems desirable to somehow "aggregate"
> the jobs of particular users on the same host (or at least on only a
> few hosts) and avoid the situation that jobs of the same user are
> distributed randomly across the cluster.
> Note that these jobs are all independent and started at possibly
> different times.
>
> What would be required is a means to influence the scheduling
> decision so that if the same user has already jobs running in
> certain queues, the job should be dispatched preferably to a
> free slot in one of these queues.
>
> Looking at the job distribution at submit time and requesting
> queues explicitly won't work, because the job distribution will
> change until the job is actually started. Defining a complex for
> every user, setting and resetting this for the queue from
> job prologue/epilogue scripts and requesting this user-specific
> complex as a soft reqirement might be feasible, but sounds messy:
> there would be many complexes needed, and users would need the
> permission to modify the queue.
>
> Does anybody with deeper GridEngine knowledge have an idea how
> this could be achieved in a reasonable way? Or is this simply
> not possible?
>
>
> Regards
> Dieter Ruppert
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=231397
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=231443

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list