[GE users] Default CPU core binding

reuti reuti at staff.uni-marburg.de
Mon Oct 18 10:38:37 BST 2010


Am 13.10.2010 um 23:00 schrieb dagru:

>> <snip>
>> How to use the load as threshold in such a situation any longer? Besides the native implementation being the length of the run queue, in my opinion there is something missing and a need to have something new, either "load by SGE jobs" (on SGE level) or "load per core" (on a Linux level). For now you can't tell much about the situation of a machine when you look only at the load.
> 
> That's interesting also for the case of queue sort method load.

Yes, good remark. For now the queue_sort_method is "load" or "seq_no,load" (although the name for the setting is only "seq_no"), where "load" refers to "host-load" as it's getting only one load value per host. This could be changed in newer versions of SGE to have the wording "host-load", because ...


> I would agree that in such a situation the load per host makes not 
> much sense. IMHO when all jobs are bound on an execution host even 
> the load per OS processor would not be from interest, because the
> different run queue lengths wouldn't have much influence to the 
> computational throughput of other cores (with the exception of the 
> automatic overclocking feature of some processors).

... you could get a load per OS processor and sort by "core-load" instead of "host-load", because there are situations where not all processes are locked to a core, and so you could handle the "D" state processes. I.e. try to get the least loaded cores inside the cluster for example.


> The bottleneck could be a shared resource (like high level caches and
> I/O). Therefore I could imagine that the average amount of waiting (for
> I/O) processes per host could be better as a queue load sort method for
> example. Would this make more sense?

Yes, when this particular load is a concern for the job in question. Means: instead of requesting something like a boolean complex BIG_IO or alike (where you estimate that this job has a high impact on I/O), it could be a way top specify that for this job the I/O wait time is of concern. In the essence, we would have a different sort oder for each type of job (the one specified in the scheduler would then be the default).

$ qsub -sort +wait_io,-tmp_free demo.sh

this job would sort the free cores first by wait_io (get least loaded one) (sort ascending), and inside the cores with the same wait_io time to get the one with the most disk space available (sort descending).

-- Reuti


> Daniel
> 
>> 
>> -- Reuti 
>> 
>> 
>>> Hristo
>>> 
>>>> -- Reuti
>>>> 
>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Chris
>>>>> 
>>>>> 
>>>>> --
>>>>> Dr Chris Jewell
>>>>> Department of Statistics
>>>>> University of Warwick
>>>>> Coventry
>>>>> CV4 7AL
>>>>> UK
>>>>> Tel: +44 (0)24 7615 0778
>>>>> 
>>>>> ------------------------------------------------------
>>>>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=285595
>>>>> 
>>>>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>>>>> 
>>>> 
>>>> ------------------------------------------------------
>>>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=286632
>>>> 
>>>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>>> 
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=286693
>>> 
>>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>>> 
>> 
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=286700
>> 
>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=286946
> 
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=288120

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list