[GE users] optimize job distribution for license usage?
dieter_ruppert at siemens.com
Mon Dec 7 14:47:34 GMT 2009
>> We use this on single processor machines only at the moment, but will
>> employ quadcore machines in a few months. So I am beginning to think
>> about how to setup this to avoid license fragmentation and to make
>> better use of the available licenses.
>> Every job requires one execution unit (core or single processor, at
>> moment), and as we have as many licenses as we have (single processor)
>> hosts, we don't need a real license check. This will change with
>> the quadcore machines, however.
>> Could such a "license aggregation" be part of a license check
>if you setup a PE with fixed allocation_rule 4 and run only parallel
>jobs it's like before (means you can only run in parallel with
>4-8-12-... cores). Every process on a machine will use all available
>cores with one license. It only becomes tricky, when you also want to
>run serial jobs or jobs with only 2 processes. What will happen to a
>job, when no license is available - it will just exit?
In our current setup, this does not happen normally. But I have provided
a "exit 99" - workaround and a load sensor looking at the available
licenses. But I know that this has to be solved more reliably.
>>> First idea could be to use a fixed alloaction_rule in your PE setup,
>>> i.e. enter a plain number there like 4 or 2 to lower fragmentation.
>> We don't use PEs at the moment, but yes, I see that somehow bundling
>> jobs together could be useful. A problem with this is that a
>> user's jobs are not always startet at the same time. Is it possible
>> to somehow "add" a job to the jobs already running in a PE, if there
>> are free slots?
>Depending on your workflow: you could submit an advance reservation
>for a complete machine by requesting 4 slots by a PE which you can
>call "smp" and allocation_rule $pe_slots, and then submit serial jobs
>into this advance reservation, as you know you have a license for sure.
I see that I have to read up more on PEs and advance reservation; I have
only vague ideas how to adapt our serial approach to this.
To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
More information about the gridengine-users