[GE users] Help needed with schedule priorities

reuti reuti at staff.uni-marburg.de
Thu Nov 19 17:27:16 GMT 2009


Am 19.11.2009 um 17:09 schrieb fx:

> ffelix <ffelixr at gmail.com> writes:
>
>> Hello Reuti,
>>
>> Yes, I understand the philosophy of SGE with respect to queues. The
>> problem is that I think gLite middleware was initially designed to
>> work with Torque where priorities are assigned to queues and the user
>> chooses the queue.
>
> We have the same situation, having to accept LCG jobs, and I  
> objected to
> having to set up a special queue, but that's the least of the ways  
> that
> stuff insists on breaking reasonable policies on your cluster.   
> (Also it
> doesn't help communication that everything seems to have a
> non-conventional name in that world.)
>
>> Later, a SGE adaptor appeared and we moved to it
>> because SGE solved more problems for us than Torque but now we are
>> facing this limitation that we don't know how to overcome
>
> I'm not clear quite what the problem is and why the normal SGE policy
> mechanisms don't do the job.  Assuming you can define a special queue
> for these jobs, you can set a priority

"priority" in the queue definition is only the "nice" value for the  
started jobs. Maybe the history is (similar to load_thresholds) big  
SMP machines with 256 or so cores. Then you can put more jobs on the  
machine than nodes and give the important jobs a higher nice value  
while all are running simultaneously.

The "weight_priority" in the scheduler configuration refers to the  
one you attach to the jobs with "qsub -p <priority> ...".

In SGE there is no "priority" on queues just based on a setting. In  
SGE jobs, projects, users,departments have a weight influencing the  
scheduling.

==

Maybe an option for the original request would be to use projects  
(and change in the adaptor the -q to -P and supply the name of the  
project instead of the queue - I don't know how the adaptor works.  
But then also changing it to -l and request a boolean complex could  
be done).

-- Reuti


> on it and have the scheduling
> policy take that into account with sufficient weight.  If all else
> fails, you can do arbitrary things with a JSV for submissions to that
> queue in a recent-enough SGE.  (Managing a queue may be easier than
> managing the large mess of VOMS(?) ids that we get here.)  I'm
> interested to understand this in case there's something we've missed.
>
>> Maybe the solution is to modify the adaptor that, at the end, is what
>> launches the qsub commands...
>
> It would be worth publishing if you can provide something  
> acceptable to
> the glite world generally.
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=228002
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=228023

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list