[GE users] Array jobs & job priorities

reuti reuti at staff.uni-marburg.de
Wed Dec 3 17:53:47 GMT 2008


Hi Oliver,

Am 03.12.2008 um 00:59 schrieb Brendon Oliver:

> I've got a scenario here that I'm not sure how to deal with, so  
> hopefully
> someone can make some suggestions (even if what I want to be able  
> to do just
> isn't possible - just so I know for sure):
>
> I have one queue on which two different (and unrelated) types of  
> jobs can be
> run.  Call them job A & job B.  They can't be run on separate  
> queues due to
> resource constraints.  The queue has multiple slots, because there  
> are other
> job types that can run on the same queue in parallel.  However job  
> types A &
> B must never be run at the same time on the same node, so each gets  
> submitted
> with a hard resource requirement ( "-l xxx=1" where xxx is the name  
> of a
> consumable complex we have set up).
>
> Job type A is much more important than B.  So we use 'qsub -p  
> 100 ...' when it
> is submitted (job type B gets no '-p' priority whent submitted).

personally I would prefer -100 and 0 instead of positive priorities,  
but if the user has the right to use priorities over 0 it's okay of  
course.

> Now job type B can often be an extremely large long-running job.   
> We know this
> in advance, so when this happens, the job gets submitted as an  
> array job (
> qsub -t 1-n ... etc.).  There will frequently be more array tasks  
> than there
> are available machines in the cluster.
>
> What I want / need / would like to happen: if an array job of type  
> B is
> currently running on the queue (eg. there might be 4x machines in  
> the queue
> cluster, tasks 1 thru 4 are executing on those 4 boxes, tasks 5  
> thru 10 are
> sitting in the queue as "pending"), if a job of type A is  
> submitted, that it
> gets scheduled before the next task of job type B gets scheduled.   
> The type A
> job _must_ get executed before the remaining tasks for job B are run.

Exactly this is what I observe - hence to do. Which SGE version are  
you using? You can switch on "report_pjob_tickets TRUE" to check the  
computed prioritzy, but switch it off in long term as it's time  
consuming.

-- Reuti

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=90920

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list