[GE users] optimize job distribution for license usage?

ruppert dieter_ruppert at siemens.com
Mon Dec 7 14:47:34 GMT 2009


>> <snip>
>> We use this on single processor machines only at the moment, but will
>> employ quadcore machines in a few months. So I am beginning to think
>> about how to setup this to avoid license fragmentation and to make
>> better use of the available licenses.
>> Every job requires one execution unit (core or single processor, at  
>> the
>> moment), and as we have as many licenses as we have (single processor)
>> hosts, we don't need a real license check. This will change with
>> the quadcore machines, however.
>> Could such a "license aggregation" be part of a license check  
>> mechanism?
>if you setup a PE with fixed allocation_rule 4 and run only parallel  
>jobs it's like before (means you can only run in parallel with  
>4-8-12-... cores). Every process on a machine will use all available  
>cores with one license. It only becomes tricky, when you also want to  
>run serial jobs or jobs with only 2 processes. What will happen to a  
>job, when no license is available - it will just exit?

In our current setup, this does not happen normally. But I have provided
a "exit 99" - workaround and a load sensor looking at the available 
licenses. But I know that this has to be solved more reliably.

>>> First idea could be to use a fixed alloaction_rule in your PE setup,
>>> i.e. enter a plain number there like 4 or 2 to lower fragmentation.
>> We don't use PEs at the moment, but yes, I see that somehow bundling
>> jobs together could be useful. A problem with this is that a  
>> particular
>> user's jobs are not always startet at the same time. Is it possible
>> to somehow "add" a job to the jobs already running in a PE, if there
>> are free slots?
>Depending on your workflow: you could submit an advance reservation  
>for a complete machine by requesting 4 slots by a PE which you can  
>call "smp" and allocation_rule $pe_slots, and then submit serial jobs  
>into this advance reservation, as you know you have a license for sure.

I see that I have to read up more on PEs and advance reservation; I have
only vague ideas how to adapt our serial approach to this. 

Dieter Ruppert


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list