[GE users] priorities (for array jobs)

Shannon V. Davidson svdavidson at swbell.net
Fri May 14 15:24:07 BST 2004


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

SGE 5.x uses job priority (-p priority) as a simple job ordering mechanism.

SGEEE 5.x uses job priority (-p priority) for ranking jobs within the 
functional and share tree policies.

SGE 6.0 unifies the GE and GEEE priority mechanisms by supporting both 
and allowing the SGE manager to determine the relative weighting or 
influence of each on the overall priority of the job.  The SGEEE 
mechanism is now supported via the new -js job_share qsub option.

Here's an excerpt from the preliminary SGE 6.0 qsub man page:

       -p priority
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Defines  or  redefines  the  priority of the job relative 
to other jobs.
              Priority is an integer in the range -1023 to 1024.  The 
default priority
              value for jobs is 0.

              Users  may  only  decrease the priority of their jobs.  
Grid Engine man­
              agers and administrators may also increase the priority 
associated  with
              jobs.  If  a pending job has higher priority, it is 
earlier eligible for
              being dispatched by the Grid Engine scheduler.

       -js job_share
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Defines or redefines the job share of the job relative  
to  other  jobs.
              Job share is an unsigned integer value.  The default job 
share value for
              jobs is 0.

              The job share influences the Share Tree Policy and the  
Functional  Pol­
              icy.  It  has  no  effect  on  the  Urgency  and  Override 
Policies (see
              share_tree(5), sched_conf(5) and the Grid Engine 
Installation and Admin­
              istration Guide for further information on the resource 
management poli­
              cies supported by Grid Engine).

              In case of the Share Tree Policy, users can distribute  
the  tickets  to
              which  they  are  currently  entitled,  among their jobs 
using different
              shares assigned via -js. If all jobs have the same job 
share value,  the
              tickets are distributed evenly. Otherwise, jobs receive 
tickets relative
              to the different job shares. Job shares are treated like  
an  additional
              level in the share tree in the latter case.

              In  connection  with the Functional Policy, the job share 
can be used to
              weight jobs within the functional job category.  Tickets 
are distributed
              relative to any uneven job share distribution treated as a 
virtual share
              distribution level underneath the functional job category.

              If both the Share Tree and the Functional Policy  are  
active,  the  job
              shares  will  have  an  effect in both policies and the 
tickets indepen­
              dently derived in each of them are added up to the total 
number of tick­
              ets for each job.


Shannon



Ron Chen wrote:

>I think SGE 6.0 has POSIX representation of priority,
>which will allow the user to have +ve or -ve priority.
>
>Shannon can tell you more about this, or just search
>Shannon's mail from the dev list archive.
>
> -Ron
>
>--- Charu Chaubal <Charu.Chaubal at Sun.COM> wrote:
>  
>
>>Hi Bryan,
>>
>>In general, you should use SGEEE priority policies
>>to implement the 
>>kind of priorities you are talking about.  This
>>means specifically the 
>>ticket-based policies of SGEEE.  Please see these
>>two HOWTOs on setting 
>>up priorities with SGEEE policies:
>>
>>
>>    
>>
>http://gridengine.sunsource.net/project/gridengine/howto/geee.html
>  
>
>>http://www.sun.com/blueprints/0703/817-3179.pdf
>>
>>
>>Regards,
>>	Charu
>>
>>On May 13, 2004, at 8:52 AM, Bryan Bayerdorffer
>>wrote:
>>
>>    
>>
>>>So I've traced the second problem some more.  In
>>>      
>>>
>>addition to the prec 
>>    
>>
>>>resource, both array jobs also request an ncv
>>>      
>>>
>>resource.  Both are 
>>    
>>
>>>consumable integers.  Suppose two array jobs are
>>>      
>>>
>>submitted in this 
>>    
>>
>>>order:
>>>
>>>qsub -t ... -l ncv=1,prec=0 ...
>>>qsub -t ... -l ncv=1,prec=1 ...
>>>
>>>Now ncv is always fully allocated.  When some jobs
>>>      
>>>
>>finish and free up 
>>    
>>
>>>ncv instances, the scheduler launches as many
>>>      
>>>
>>tasks from the first 
>>    
>>
>>>array job as it can.  Then ncv==0 and so no tasks
>>>      
>>>
>>from the second 
>>    
>>
>>>array job can be launched.
>>>
>>>So what I want is some way to tell the scheduler,
>>>      
>>>
>>"Fill the requests 
>>    
>>
>>>for prec before the requests for ncv".  Is there a
>>>      
>>>
>>way to do this?  
>>    
>>
>>>Would just listing prec first in the resource
>>>      
>>>
>>request do this 
>>    
>>
>>>reliably?
>>>
>>>Any help on the priority question?
>>>
>>>
>>>Bryan Bayerdorffer wrote:
>>>      
>>>
>>>>Two questions (SGEEE v5.3p6)
>>>>- Users can't submit jobs with positive priority
>>>>        
>>>>
>>values or qalter 
>>    
>>
>>>>pending jobs to higher priorities.  qsub -p 1 is
>>>>        
>>>>
>>silently ignored and 
>>    
>>
>>>>the job gets priority 0.  qalter -p 1 yields
>>>>        
>>>>
>>'denied: "user" must be 
>>    
>>
>>>>operator to increase job priority'.  Is there a
>>>>        
>>>>
>>way to enable all 
>>    
>>
>>>>users to set positive priorities without giving
>>>>        
>>>>
>>them other operator 
>>    
>>
>>>>privileges?
>>>>- Each host has a queue for normal jobs and a
>>>>        
>>>>
>>queue for urgent jobs.  
>>    
>>
>>>>The urgent queues offer a resource prec=1 while
>>>>        
>>>>
>>the normal queues 
>>    
>>
>>>>offer prec=0. When all the normal queues are full
>>>>        
>>>>
>>and I submit some 
>>    
>>
>>>>more jobs that request prec=0, followed by some
>>>>        
>>>>
>>jobs that request 
>>    
>>
>>>>prec=1 (same priority for all), my urgent
>>>>        
>>>>
>>(prec=1) jobs wait behind 
>>    
>>
>>>>my normal jobs, even though they could be
>>>>        
>>>>
>>dispatched.  Shouldn't the 
>>    
>>
>>>>scheduler detect this and allow my urgent jobs to
>>>>        
>>>>
>>run, even though 
>>    
>>
>>>>they were submitted after my normal jobs?
>>>>        
>>>>
>>>-- 
>>> .. ..-. ..- -.-. .- -. .-. . .- -.. - .... .. ...
>>>      
>>>
>>--. . - .- .-.. .. 
>>    
>>
>>>..-. .
>>>Bryan Bayerdorffer        bryan at meatspace.net     
>>>      
>>>
>>     
>>    
>>
>>>bryan at spd.analog.com
>>>                     (Wit's End Computation
>>>      
>>>
>>Center)       (Analog 
>>    
>>
>>>Devices)
>>>
>>>"'National Security' is the root password to the
>>>      
>>>
>>Constitution."
>>    
>>
>>>                                                  
>>>      
>>>
>>             -- 
>>    
>>
>>>Phil Karn
>>>
>>>
>>>      
>>>
>---------------------------------------------------------------------
>  
>
>>>To unsubscribe, e-mail:
>>>      
>>>
>>users-unsubscribe at gridengine.sunsource.net
>>    
>>
>>>For additional commands, e-mail:
>>>      
>>>
>>users-help at gridengine.sunsource.net
>>    
>>
>>>      
>>>
>########################################################
>  
>
>># Charu V. Chaubal				# Phone: (650) 786-7672
>>(x87672)
>># Grid Computing Technologist	# Fax:   (650)
>>786-4591
>># Sun Microsystems, Inc.			# Email:
>>charu.chaubal at sun.com
>>
>>    
>>
>########################################################
>  
>
>>    
>>
>
>
>
>	
>		
>__________________________________
>Do you Yahoo!?
>SBC Yahoo! - Internet access at a great low price.
>http://promo.yahoo.com/sbc/
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>
>  
>

-- 
___________________________________________

Shannon V. Davidson <svdavidson at swbell.net>
Senior Software Engineer           Raytheon
636-479-7465 office        443-383-0331 fax
___________________________________________





More information about the gridengine-users mailing list