[GE users] optimize job distribution for license usage?

reuti reuti at staff.uni-marburg.de
Mon Dec 7 12:49:07 GMT 2009


Hi,

Am 07.12.2009 um 10:05 schrieb ruppert:

> <snip>
> We use this on single processor machines only at the moment, but will
> employ quadcore machines in a few months. So I am beginning to think
> about how to setup this to avoid license fragmentation and to make
> better use of the available licenses.
>
> Every job requires one execution unit (core or single processor, at  
> the
> moment), and as we have as many licenses as we have (single processor)
> hosts, we don't need a real license check. This will change with
> the quadcore machines, however.
>
> Could such a "license aggregation" be part of a license check  
> mechanism?

if you setup a PE with fixed allocation_rule 4 and run only parallel  
jobs it's like before (means you can only run in parallel with  
4-8-12-... cores). Every process on a machine will use all available  
cores with one license. It only becomes tricky, when you also want to  
run serial jobs or jobs with only 2 processes. What will happen to a  
job, when no license is available - it will just exit?


>> First idea could be to use a fixed alloaction_rule in your PE setup,
>> i.e. enter a plain number there like 4 or 2 to lower fragmentation.
>>
>
> We don't use PEs at the moment, but yes, I see that somehow bundling
> jobs together could be useful. A problem with this is that a  
> particular
> user's jobs are not always startet at the same time. Is it possible
> to somehow "add" a job to the jobs already running in a PE, if there
> are free slots?

Depending on your workflow: you could submit an advance reservation  
for a complete machine by requesting 4 slots by a PE which you can  
call "smp" and allocation_rule $pe_slots, and then submit serial jobs  
into this advance reservation, as you know you have a license for sure.

-- Reuti

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=232004

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list