[GE users] priority adjustment ...

Andreas.Haas at Sun.COM Andreas.Haas at Sun.COM
Tue Feb 27 17:39:07 GMT 2007


Ok. That means only the urgency policy is in use.
Next you do

    # qstat -urg

in the older clusters to figure out what aspects
of the urgency policy are actualy used. If you
find there neither deadline nor waiting time you
should then do a

    # qstat -r

as this will tell you the urgency contribution of each 
resource in your older cluster.

Andreas


On Tue, 27 Feb 2007, Lydia Heck wrote:

>
>
>
> qstat -pri on the changing priorities cluster looks as follows
>
> oberon# qstat -pri
> job-ID  prior   nurg    npprior ntckts   ppri name       user         state submit/start at     queue                          slots ja-task-ID
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2078                   1 9
>  19486 0.04680 0.41803 0.50000 0.50000     0 DEFL25N512 caius        r     02/25/2007 16:30:19 cordelia.q at m2090                 128
>  16441 0.10500 1.00000 0.50000 0.50000     0 GimicB     tt           r     02/25/2007 19:35:10 cordelia.q at m2094                 256
>  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2103                   1 4
>  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2107                   1 10
>  21183 0.01316 0.08163 0.50000 0.50000     0 C9_14_39   arj          r     02/27/2007 09:03:45 cordelia.q at m2119                  32
>  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2152                   1 6
>  21207 0.00531 0.00312 0.50000 0.50000     0 SliceFoF   rcrain       r     02/27/2007 10:27:45 cordelia.q at m2223                   1 8
>
> .....
>
>
> on the new cluster it shows ...
>
> job-ID  prior   nurg    npprior ntckts   ppri name       user         state submit/start at     queue                          slots ja-task-ID
> ----------------------------------------------------------------------------------------------------------------------------------------------------
>   3077 0.55500 0.50000 0.50000 0.50000     0 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm26.phyastcl.du     1 356
>   3077 0.55500 0.50000 0.50000 0.50000     0 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm26.phyastcl.du     1 357
>   3036 0.55500 0.50000 0.50000 0.50000     0 ReCat.csh  cai          r     02/27/2007 13:05:57 extrashort.q at cdm3.phyastcl.dur     1
>   3075 0.55500 0.50000 0.50000 0.50000     0 galaxy_tre jch          r     02/27/2007 14:16:54 extrashort.q at cdm33.phyastcl.du     1 338
>   3076 0.55500 0.50000 0.50000 0.50000     0 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm33.phyastcl.du     1 339
>   2906 0.55500 0.50000 0.50000 0.50000     0 xi.0.1_470 irpn         r     02/27/2007 15:23:39 medium.q at cdm10.phyastcl.dur.ac     1 85
>   2906 0.55500 0.50000 0.50000 0.50000     0 xi.0.1_470 irpn         r     02/27/2007 15:23:39 medium.q at cdm10.phyastcl.dur.ac     1 86
>   2906 0.55500 0.50000 0.50000 0.50000     0 xi.0.1_470 irpn         r     02/27/2007 10:56:27 medium.q at cdm11.phyastcl.dur.ac     1 29
>
> ........
>
> On Tue, 27 Feb 2007 Andreas.Haas at sun.com wrote:
>> Thanks. The sched_conf(5) seems to be fine at large
>>
>>    weight_priority                   1.000000
>>    weight_urgency                    0.100000
>>    weight_ticket                     0.010000
>>    weight_waiting_time               0.000000
>>    weight_deadline                   3600000.000000
>>
>> I suggest you do a
>>
>>     # qstat -pri
>>
>> as to figure out where the priorities actually come from in the
>> cluster where you see them. This should bring us closer to
>> the problem.
>>
>> Andreas
>>
>> On Tue, 27 Feb 2007, Lydia Heck wrote:
>>
>>>
>>> Hi Andreas,
>>>
>>> the prioritisation I seen on one of my clusters is something like
>>>
>>> 16308 0.08690 debug      dph0elh      r     02/18/2007 17:38:28 miranda.q at m2007   128
>>>  21178 0.01963 KinghMov   dph3nlm      r     02/26/2007 23:53:15 miranda.q at m2009                   46
>>>  16320 0.05441 CB-Gh4038  simulate     r     02/19/2007 17:02:13 miranda.q at m2014                   16
>>>  19670 0.02297 Kinghalo   dph3nlm      r     02/26/2007 14:07:18 miranda.q at m2043                   50
>>>  16322 0.05441 CB-Gh7252  simulate     r     02/19/2007 17:03:13 quintor.q at m1005                   16
>>>  21232 0.00880 halo-9_14_ jch          r     02/27/2007 15:01:59 quintor.q at m1029                   16
>>>  16611 0.03751 CB-GC05    simulate     r     02/22/2007 15:26:04 quintor.q at m1042                   16
>>>  16609 0.03751 CB-GC0     simulate     r     02/22/2007 15:25:50 quintor.q at m1186                   16
>>>  16610 0.03751 CB-GC02    simulate     r     02/22/2007
>>> .....
>>> the priorities are different and adjusted by the system.
>>>
>>> on my new set up of another cluster is see
>>>
>>> -----------------------------------------------------------------------------------------------------------------
>>>   3077 0.55500 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm26.phyastcl.du     1 356
>>>   3077 0.55500 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm26.phyastcl.du     1 357
>>>   3036 0.55500 ReCat.csh  cai          r     02/27/2007 13:05:57 extrashort.q at cdm3.phyastcl.dur     1
>>>   3075 0.55500 galaxy_tre jch          r     02/27/2007 14:16:54 extrashort.q at cdm33.phyastcl.du     1 338
>>>   3076 0.55500 galaxy_tre jch          r     02/27/2007 14:17:09 extrashort.q at cdm33.phyastcl.du     1 339
>>>   2906 0.55500 xi.0.1_470 irpn         r     02/27/2007 15:23:39 medium.q at cdm10.phyastcl.dur.ac     1 85
>>>   2906 0.55500 xi.0.1_470 irpn         r     02/27/2007 15:23:39
>>>
>>> All the priorities are the same.
>>>
>>>
>>> I compared the configurations of both and I could not see any difference ..
>>>
>>>
>>>
>>> the output of qconf -ssconf is
>>>
>>> algorithm                         default
>>> schedule_interval                 0:0:15
>>> maxujobs                          0
>>> queue_sort_method                 load
>>> job_load_adjustments              np_load_short=0.5
>>> load_adjustment_decay_time        0:2:00
>>> load_formula                      np_load_avg
>>> schedd_job_info                   true
>>> flush_submit_sec                  0
>>> flush_finish_sec                  0
>>> params                            none
>>> reprioritize_interval             0:5:00
>>> halftime                          168
>>> usage_weight_list                 cpu=1.000000,mem=0.000000,io=0.000000
>>> compensation_factor               5.000000
>>> weight_user                       0.250000
>>> weight_project                    0.250000
>>> weight_department                 0.250000
>>> weight_job                        0.250000
>>> weight_tickets_functional         10000
>>> weight_tickets_share              0
>>> share_override_tickets            TRUE
>>> share_functional_shares           TRUE
>>> max_functional_jobs_to_schedule   200
>>> report_pjob_tickets               TRUE
>>> max_pending_tasks_per_job         100
>>> halflife_decay_list               none
>>> policy_hierarchy                  OFS
>>> weight_ticket                     0.010000
>>> weight_waiting_time               0.000000
>>> weight_deadline                   3600000.000000
>>> weight_urgency                    0.100000
>>> weight_priority                   1.000000
>>> max_reservation                   0
>>> default_duration                  0:10:0
>>>
>>>
>>> On Tue, 27 Feb 2007 Andreas.Haas at sun.com wrote:
>>>
>>>> On Tue, 27 Feb 2007, Lydia Heck wrote:
>>>>
>>>>>
>>>>> SGE: 6.09
>>>>>
>>>>> Although the users have now been more than 3000 jobs on my new setup,
>>>>> the priority
>>>>>
>>>>> 0.55500
>>>>>
>>>>> stays the same for all of the users.
>>>>> Should I expect that?
>>>>>
>>>>>
>>>>> The scheduler is configured with default values. I am using default
>>>>> values on another cluster, which runs  SGE 6.07
>>>>
>>>> Lydia, what kind of prioritization you hope to see and what
>>>> exactly you get with
>>>>
>>>>     # qconf -ssconf
>>>>
>>>> Thanks,
>>>> Andreas
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>>>
>>>
>>> ------------------------------------------
>>> Dr E L  Heck
>>>
>>> University of Durham
>>> Institute for Computational Cosmology
>>> Ogden Centre
>>> Department of Physics
>>> South Road
>>>
>>> DURHAM, DH1 3LE
>>> United Kingdom
>>>
>>> e-mail: lydia.heck at durham.ac.uk
>>>
>>> Tel.: + 44 191 - 334 3628
>>> Fax.: + 44 191 - 334 3645
>>> ___________________________________________
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>
> ------------------------------------------
> Dr E L  Heck
>
> University of Durham
> Institute for Computational Cosmology
> Ogden Centre
> Department of Physics
> South Road
>
> DURHAM, DH1 3LE
> United Kingdom
>
> e-mail: lydia.heck at durham.ac.uk
>
> Tel.: + 44 191 - 334 3628
> Fax.: + 44 191 - 334 3645
> ___________________________________________
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list