[GE users] optimize job distribution for license usage?

ruppert dieter_ruppert at siemens.com
Mon Dec 7 09:05:34 GMT 2009


    [ The following text is in the "utf-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Hi,


>
>> we want to use an application with GridEngine which is licensed
>> according to "user at host": starting the application multiple times
>> by the same user on the same host requires only one license token,
>> whereas another user on the same host (or the same user on another
>> host) requires additional license tokens.
>
>interesting. This would imply that you get more from the license if  
>you buy a single machine with eight installed quad-cores, instead of  
>buying eight machines with two dual-cores installed each.

Yes.

>
>a) How many cores per machine you have right now?
>
>b) How man cores the users usually request for their jobs?
>
>c) How is the limit of the licenses checked? SGE needs to know  
>beforehand already, whether the job can be scheduled at all.
>

We use this on single processor machines only at the moment, but will
employ quadcore machines in a few months. So I am beginning to think
about how to setup this to avoid license fragmentation and to make
better use of the available licenses.

Every job requires one execution unit (core or single processor, at the 
moment), and as we have as many licenses as we have (single processor)
hosts, we don't need a real license check. This will change with
the quadcore machines, however.

Could such a "license aggregation" be part of a license check mechanism?   


>First idea could be to use a fixed alloaction_rule in your PE setup,  
>i.e. enter a plain number there like 4 or 2 to lower fragmentation.
>

We don't use PEs at the moment, but yes, I see that somehow bundling
jobs together could be useful. A problem with this is that a particular
user's jobs are not always startet at the same time. Is it possible 
to somehow "add" a job to the jobs already running in a PE, if there
are free slots?

D. Ruppert   



>-- Reuti
>
>
>> Assume that there are two hosts A and B, each capable of running
>> two jobs in parallel, i.e. four jobs could run at the same time.
>> If two users each submit two jobs, this would require two license
>> tokens if the jobs of one user both end up on host A, and the jobs
>> of the other user on host B. Four license tokens would be needed
>> if the two jobs of each user end up on different hosts.
>>
>> From this licensing scheme, it seems desirable to somehow "aggregate"
>> the jobs of particular users on the same host (or at least on only a
>> few hosts) and avoid the situation that jobs of the same user are
>> distributed randomly across the cluster.
>> Note that these jobs are all independent and started at possibly
>> different times.
>>
>> What would be required is a means to influence the scheduling
>> decision so that if the same user has already jobs running in
>> certain queues, the job should be dispatched preferably to a
>> free slot in one of these queues.
>>
>> Looking at the job distribution at submit time and requesting
>> queues explicitly won't work, because the job distribution will
>> change until the job is actually started. Defining a complex for
>> every user, setting and resetting this for the queue from
>> job prologue/epilogue scripts and requesting this user-specific
>> complex as a soft reqirement might be feasible, but sounds messy:
>> there would be many complexes needed, and users would need the
>> permission to modify the queue.
>>
>> Does anybody with deeper GridEngine knowledge have an idea how
>> this could be achieved in a reasonable way? Or is this simply
>> not possible?
>>
>>
>> Regards
>> Dieter Ruppert
>>
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do? 
>> dsForumId=38&dsMessageId=231397
>>
>> To unsubscribe from this discussion, e-mail: [users- 
>> unsubscribe at gridengine.sunsource.net].
>
>------------------------------------------------------
>http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=2314
43
>
>To unsubscribe from this discussion, e-mail: 
[users-unsubscribe at gridengine.sunsource.net].

----------------------------------
ePS & RTS Automation Software GmbH
Benzstr. 1
D-71272 Renningen
Geschäftsführer: Gernot Kral, Frank Lubnau
Sitz der Gesellschaft: Renningen
Registergericht: Leonberg HRB 253220

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=231971

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list