[GE users] priority adjustment ...

Lydia Heck lydia.heck at durham.ac.uk
Tue Feb 27 18:35:01 GMT 2007


I find the following

ob-ID  prior   nurg    urg      rrcontr  wtcontr  dlcontr  name       user         state submit/start at      deadline           queue                          slots ja-task-ID
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  21207 0.00531 0.00312     8336     1000     7336        0 SliceFoF   rcrain       r     02/27/2007 10:27:45                     cordelia.q at m2078                   1 9
  19486 0.04680 0.41803   179586   128000    51586        0 DEFL25N512 caius        r     02/25/2007 16:30:19                     cordelia.q at m2090                 128

And I am puzzled. I never did configure for urgency and deadlines.

Lydia


 On Tue, 27 Feb 2007 Andreas.Haas at sun.com wrote:

> Ok. That means only the urgency policy is in use.
> Next you do
>
>     # qstat -urg
>
> in the older clusters to figure out what aspects
> of the urgency policy are actualy used. If you
> find there neither deadline nor waiting time you
> should then do a
>
>     # qstat -r
>
> as this will tell you the urgency contribution of each
> resource in your older cluster.
>
> Andreas
>
>
> On Tue, 27 Feb 2007, Lydia Heck wrote:
>
> >
> >
> >
> > qstat -pri on the changing priorities cluster looks as follows
> >
> > oberon# qstat -pri
> > job-ID  prior   nurg    npprior ntckts   ppri name       user         state submit/start at     queue                          slots ja-task-ID
> > ----------------------------------------------------------------------------------------------------------------------------------------------------
> >  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2078                   1 9
> >  19486 0.04680 0.41803 0.50000 0.50000     0 DEFL25N512 caius        r     02/25/2007 16:30:19 cordelia.q at m2090                 128
> >  16441 0.10500 1.00000 0.50000 0.50000     0 GimicB     tt           r     02/25/2007 19:35:10 cordelia.q at m2094                 256
> >  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2103                   1 4
> >  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2107                   1 10
> >  21183 0.01316 0.08163 0.50000 0.50000     0 C9_14_39   arj          r     02/27/2007 09:03:45 cordelia.q at m2119                  32
> >  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2152                   1 6
> >  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2223                   1 8
> >
> > .....
> >
> >
> > on the new cluster it shows ...
> >
> > job-ID  prior   nurg    npprior ntckts   ppri name       user         state submit/start at     queue                          slots ja-task-ID
> > ----------------------------------------------------------------------------------------------------------------------------------------------------
> >   3077 0.55500 0.50000 0.50000 0.50000     0 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm26.phyastcl.du     1 356
> >   3077 0.55500 0.50000 0.50000 0.50000     0 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm26.phyastcl.du     1 357
> >   3036 0.55500 0.50000 0.50000 0.50000     0 ReCat.csh  cai          r     02/27/2007 13:05:57 extrashort.q at cdm3.phyastcl.dur     1
> >   3075 0.55500 0.50000 0.50000 0.50000     0 galaxy_tre jch          r     02/27/2007 14:16:54 extrashort.q at cdm33.phyastcl.du     1 338
> >   3076 0.55500 0.50000 0.50000 0.50000     0 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm33.phyastcl.du     1 339
> >   2906 0.55500 0.50000 0.50000 0.50000     0 xi.0.1_470 irpn         r     02/27/2007 15:23:39 medium.q at cdm10.phyastcl.dur.ac     1 85
> >   2906 0.55500 0.50000 0.50000 0.50000     0 xi.0.1_470 irpn         r     02/27/2007 15:23:39 medium.q at cdm10.phyastcl.dur.ac     1 86
> >   2906 0.55500 0.50000 0.50000 0.50000     0 xi.0.1_470 irpn         r     02/27/2007 10:56:27 medium.q at cdm11.phyastcl.dur.ac     1 29
> >
> > ........
> >
> > On Tue, 27 Feb 2007 Andreas.Haas at sun.com wrote:
> >> Thanks. The sched_conf(5) seems to be fine at large
> >>
> >>    weight_priority                   1.000000
> >>    weight_urgency                    0.100000
> >>    weight_ticket                     0.010000
> >>    weight_waiting_time               0.000000
> >>    weight_deadline                   3600000.000000
> >>
> >> I suggest you do a
> >>
> >>     # qstat -pri
> >>
> >> as to figure out where the priorities actually come from in the
> >> cluster where you see them. This should bring us closer to
> >> the problem.
> >>
> >> Andreas
> >>
> >> On Tue, 27 Feb 2007, Lydia Heck wrote:
> >>
> >>>
> >>> Hi Andreas,
> >>>
> >>> the prioritisation I seen on one of my clusters is something like
> >>>
> >>> 16308 0.08690 debug      dph0elh      r     02/18/2007 17:38:28 miranda.q at m2007   128
> >>>  21178 0.01963 KinghMov   dph3nlm      r     02/26/2007 23:53:15 miranda.q at m2009                   46
> >>>  16320 0.05441 CB-Gh4038  simulate     r     02/19/2007 17:02:13 miranda.q at m2014                   16
> >>>  19670 0.02297 Kinghalo   dph3nlm      r     02/26/2007 14:07:18 miranda.q at m2043                   50
> >>>  16322 0.05441 CB-Gh7252  simulate     r     02/19/2007 17:03:13 quintor.q at m1005                   16
> >>>  21232 0.00880 halo-9_14_ jch          r     02/27/2007 15:01:59 quintor.q at m1029                   16
> >>>  16611 0.03751 CB-GC05    simulate     r     02/22/2007 15:26:04 quintor.q at m1042                   16
> >>>  16609 0.03751 CB-GC0     simulate     r     02/22/2007 15:25:50 quintor.q at m1186                   16
> >>>  16610 0.03751 CB-GC02    simulate     r     02/22/2007
> >>> .....
> >>> the priorities are different and adjusted by the system.
> >>>
> >>> on my new set up of another cluster is see
> >>>
> >>> -----------------------------------------------------------------------------------------------------------------
> >>>   3077 0.55500 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm26.phyastcl.du     1 356
> >>>   3077 0.55500 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm26.phyastcl.du     1 357
> >>>   3036 0.55500 ReCat.csh  cai          r     02/27/2007 13:05:57 extrashort.q at cdm3.phyastcl.dur     1
> >>>   3075 0.55500 galaxy_tre jch          r     02/27/2007 14:16:54 extrashort.q at cdm33.phyastcl.du     1 338
> >>>   3076 0.55500 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm33.phyastcl.du     1 339
> >>>   2906 0.55500 xi.0.1_470 irpn         r     02/27/2007 15:23:39 medium.q at cdm10.phyastcl.dur.ac     1 85
> >>>   2906 0.55500 xi.0.1_470 irpn         r     02/27/2007 15:23:39
> >>>
> >>> All the priorities are the same.
> >>>
> >>>
> >>> I compared the configurations of both and I could not see any difference ..
> >>>
> >>>
> >>>
> >>> the output of qconf -ssconf is
> >>>
> >>> algorithm                         default
> >>> schedule_interval                 0:0:15
> >>> maxujobs                          0
> >>> queue_sort_method                 load
> >>> job_load_adjustments              np_load_short=0.5
> >>> load_adjustment_decay_time        0:2:00
> >>> load_formula                      np_load_avg
> >>> schedd_job_info                   true
> >>> flush_submit_sec                  0
> >>> flush_finish_sec                  0
> >>> params                            none
> >>> reprioritize_interval             0:5:00
> >>> halftime                          168
> >>> usage_weight_list                 cpu=1.000000,mem=0.000000,io=0.000000
> >>> compensation_factor               5.000000
> >>> weight_user                       0.250000
> >>> weight_project                    0.250000
> >>> weight_department                 0.250000
> >>> weight_job                        0.250000
> >>> weight_tickets_functional         10000
> >>> weight_tickets_share              0
> >>> share_override_tickets            TRUE
> >>> share_functional_shares           TRUE
> >>> max_functional_jobs_to_schedule   200
> >>> report_pjob_tickets               TRUE
> >>> max_pending_tasks_per_job         100
> >>> halflife_decay_list               none
> >>> policy_hierarchy                  OFS
> >>> weight_ticket                     0.010000
> >>> weight_waiting_time               0.000000
> >>> weight_deadline                   3600000.000000
> >>> weight_urgency                    0.100000
> >>> weight_priority                   1.000000
> >>> max_reservation                   0
> >>> default_duration                  0:10:0
> >>>
> >>>
> >>> On Tue, 27 Feb 2007 Andreas.Haas at sun.com wrote:
> >>>
> >>>> On Tue, 27 Feb 2007, Lydia Heck wrote:
> >>>>
> >>>>>
> >>>>> SGE: 6.09
> >>>>>
> >>>>> Although the users have now been more than 3000 jobs on my new setup,
> >>>>> the priority
> >>>>>
> >>>>> 0.55500
> >>>>>
> >>>>> stays the same for all of the users.
> >>>>> Should I expect that?
> >>>>>
> >>>>>
> >>>>> The scheduler is configured with default values. I am using default
> >>>>> values on another cluster, which runs  SGE 6.07
> >>>>
> >>>> Lydia, what kind of prioritization you hope to see and what
> >>>> exactly you get with
> >>>>
> >>>>     # qconf -ssconf
> >>>>
> >>>> Thanks,
> >>>> Andreas
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>>> For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>>>
> >>>
> >>> ------------------------------------------
> >>> Dr E L  Heck
> >>>
> >>> University of Durham
> >>> Institute for Computational Cosmology
> >>> Ogden Centre
> >>> Department of Physics
> >>> South Road
> >>>
> >>> DURHAM, DH1 3LE
> >>> United Kingdom
> >>>
> >>> e-mail: lydia.heck at durham.ac.uk
> >>>
> >>> Tel.: + 44 191 - 334 3628
> >>> Fax.: + 44 191 - 334 3645
> >>> ___________________________________________
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>> For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >> For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>
> >
> > ------------------------------------------
> > Dr E L  Heck
> >
> > University of Durham
> > Institute for Computational Cosmology
> > Ogden Centre
> > Department of Physics
> > South Road
> >
> > DURHAM, DH1 3LE
> > United Kingdom
> >
> > e-mail: lydia.heck at durham.ac.uk
> >
> > Tel.: + 44 191 - 334 3628
> > Fax.: + 44 191 - 334 3645
> > ___________________________________________
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > For additional commands, e-mail: users-help at gridengine.sunsource.net
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>

------------------------------------------
Dr E L  Heck

University of Durham
Institute for Computational Cosmology
Ogden Centre
Department of Physics
South Road

DURHAM, DH1 3LE
United Kingdom

e-mail: lydia.heck at durham.ac.uk

Tel.: + 44 191 - 334 3628
Fax.: + 44 191 - 334 3645
___________________________________________

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list