[GE users] newbie- setting queue to auto nice jobs by time

Andreas Haas Andreas.Haas at Sun.COM
Mon May 9 12:25:45 BST 2005


Hi Alan,

On Mon, 9 May 2005, Alan Carriou wrote:

> Hi
>
> The answer given in the first link you gave use a setpriority() call to
> renice the job. But, in case of parallel jobs which run on several
> computers, is seems to me that this will only change the nice value of
> the process, or the group of processes, on the host where the example
> program is run.

True. If your parallel job is tightly integrated Grid Engine knows
it's tasks.

>
> I hoped that qalter could do that, so I have looked a bit in the doc,
> but I haven't found many things regarding nice values. qalter can set a
> "priority" between -1023 and 1024, but that doesn't fit the -20 +19 unix
> nice range.

The qalter -p priority is about the dispatch priority with Grid
Engine scheduling algorithm. It's not related to runtime priorities
at all.

> The queue_conf "priority" option is meant to set the default nice value,
> but is said to have "no  effect  because  SGE  adjusts  priorities
>   dynamically to implement the SGE entitlement policy goals" (man
> queue_conf). man sched_conf tells how to change the priority adjusting
> interval, and how to disable it.

The queue_conf(5) priority does have an effect if dynamic repriotiziation
is disabled which is the default. Changes to queue_conf(5) priority however
are not propagated to execds and applied to already running jobs.

> Other places of the man pages and user/admin Guides talk about job
> priorities, regarding the way submitted jobs are sorted to choose which
> ones will be run first, but it is not clear to me if that has also a
> consequence of the nice value of a running job.

If you're using dynamic repriotiziation it should be achievable. In
this case running jobs priority is controlled with Grid Engine ticket
mechanism. As you might know there are three sources for tickets that
affect jobs total ticket amount

  - sharetree ticktes
  - functional tickets
  - override tickets

for your purpose you might try use of override tickets as this is the policy
for directly affecting job priorities without a detour. Make sure you use
in sched_conf(5)

   weight_tickets_functional         0
   weight_tickets_share              0
   reprioritize_interval             0:0:30

dynamic repriotization adjusts nice values in order to achieve the same
resource utilization (measured) ratio like with running tickets at a per
execution host basis. You can observe resource utilization and tickets
with qstat -ext.

Give it a try: Two jobs have nearly equal nice values

   # qsub -q all.q -N high $SGE_ROOT/examples/jobs/worker.sh 3600
   Your job 27 ("high") has been submitted.
   # qsub -q all.q -N low $SGE_ROOT/examples/jobs/worker.sh 3600
   Your job 28 ("low") has been submitted.

  PID USERNAME THR PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
 7787 ah114088   3  37   -9 2368K 1488K cpu/1    0:21 16.05% work
 7789 ah114088   3  40  -10 2368K 1488K cpu/0    0:21 16.01% work

until override tickets are changed

   # qalter -ot 1000000 <jobid>

  PID USERNAME THR PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
 7789 ah114088   3  30  -10 2368K 1488K cpu/2    1:06 23.98% work
 7787 ah114088   3   0   19 2368K 1488K cpu/1    1:06 23.97% work

please also see the execd_params PTF_MIN_PRIORITY and PTF_MAX_PRIORITY
in sge_conf(5). These parameters determine minimum/maximum nice value
with repriotization.

Note dynamic repriotization is not supported with all Grid Engine binary
platforms. It's merely missing with HPUX, AIX and DARWIN.

Regards,
Andreas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list