RE : [GE users] making consumable not related to PE

GARDAIS Ionel Ionel.Gardais at tech-advantage.com
Wed Jul 23 19:51:19 BST 2008


    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Hi Ravi,

Having a complex 'hds' set to 2 will not work as I expect : it will allow two process to run on the same HD.

I provide my users wrapper scripts to launch their jobs so I'll head to the "scaled complex" trick pointed by Mark using a basic "bc" calculation to get the right value.
(for curious, something like
$ qsub -pe intelmpi $REQ_CPU_CNT -l data-1-1=`echo "scale=3;1/$REQ_CPU_CNT" | bc -l` /the/job/script
with REQ_CPU_CNT set in the wrapper through the command line by user)


<quote>
Scaling the complex would mean that you want to bypass this restriction!!

If you really want to restrict the job based on the job's usage of the 
disk and make sure only one sge job is accessing the disk at a time, 
then you don't want to make it consumable rather get the value for the 
complex using a load sensor.
</quote>

Well, I'm lost with this one ...
What does a load sensor will monitor ? As disk I/O might not be constant in time, triggering a job based on this is not an option.
Or maybe doing some magic in the start_proc_arg and stop_proc_arg of the PE ?

Ionel


-------- Message d'origine--------
De: Ravi Chandra Nallan [mailto:Ravichandra.Nallan at Sun.COM]
Date: mer. 23/07/2008 20:16
?: users at gridengine.sunsource.net
Objet : Re: [GE users] making consumable not related to PE
 
Ionel GARDAIS wrote:
> Hi list,
>
> I'd like to schedule jobs based on the use (or not) of a local harddrive.
> I have an execution host with two harddrives mounted under /data-1-1 
> and /data-1-2.
You mean this exec host has 2 HDs, so ideally I would use 1 complex, 
'hds' with int type and set it to 2 for this host!
>
> I created two consumables resources :
> data-1-1            d11        INT         <=    YES         
> YES        0        0
> data-1-2            d12        INT         <=    YES         
> YES        0        0
>
> Then I bound these resources  to the execution host  :
> hostname              bulnode-1.beicip.fr
> complex_values        data-1-1=1,data-1-2=1
>
> When I submit multiple jobs to the host, requesting either for d11 or 
> d12 complexes, I have one job scheduled using resource d11 and one job 
> using d12.
> $ qsub -l d11=1 -b y /bin/sleep 30 # (run multiple times)
> $ qsub -l d12=1 -b y /bin/sleep 30 # (run multiple times)
so I can directly do a 'qsub -l hds=1 ......' need not know where it is 
mounted on!
>
> $ qstat
>  18574 0.55500 sleep      gardais      r     07/23/2008 14:45:15 
> bul.q at bulnode-1.beicip.fr      MASTER
>  18578 0.55500 sleep      gardais      r     07/23/2008 14:45:15 
> bul.q at bulnode-1.beicip.fr      MASTER
>  18575 0.55500 sleep      gardais      qw    07/23/2008 14:45:08
>  18576 0.55500 sleep      gardais      qw    07/23/2008 14:45:08
>  18577 0.55500 sleep      gardais      qw    07/23/2008 14:45:08
>  18579 0.55500 sleep      gardais      qw    07/23/2008 14:45:13
>  18580 0.55500 sleep      gardais      qw    07/23/2008 14:45:13
>  18581 0.55500 sleep      gardais      qw    07/23/2008 14:45:13
>  18582 0.55500 sleep      gardais      qw    07/23/2008 14:45:14
>
>
> Problem arise when I request a PE : job stays in qw state because it 
> require $NSLOTS to run but the complex definition is only set to "1".
Does make sense. Actual slots it would require may be <= $NSLOTS or # of 
parallel jobs to be precise, and
>
> What I want to do is to restrict the number of running job per 
> harddrive to one.
and that could be achieved using a simple complex 'hds' !
Scaling the complex would mean that you want to bypass this restriction!!

If you really want to restrict the job based on the job's usage of the 
disk and make sure only one sge job is accessing the disk at a time, 
then you don't want to make it consumable rather get the value for the 
complex using a load sensor.

Hope that helps,

regards,
~Ravi
>
> Any clues ?
>
> Thanks,
> Ionel
>
> ------------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




    [ Part 2: "Attached Text" ]

    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net



More information about the gridengine-users mailing list