[GE users] Array Jobs and Resource Allocation

Reuti reuti at staff.uni-marburg.de
Mon Nov 3 16:43:30 GMT 2008


Am 03.11.2008 um 00:16 schrieb Mag Gam:

> Great. Thanks
>
> I think the RQS you mentioned in the post makes sense. And also

Sorry for the confusion, I meant only the first part of my post, i.e.  
making virtual_free or h_vmem consumable and attaching it to the  
nodes. The RQS is only necessary, if each user should be allowed to  
use less than the available memory on each node on their own, i.e. to  
leave memory for other users on the same node free.

If he wants to run a big job, maybe it's necessary to have the option  
to request the complete memory.

> http://wiki.gridengine.info/wiki/index.php/RQS_Common_Uses here and
> http://gridengine.sunsource.net/source/browse/*checkout*/gridengine/ 
> doc/devel/rfe/ResourceQuotaSpecification.html
> are some good techniques.
>
> Basically, with RQS I am planning to do
>
> {
>  name mem_limit
> enabled true
> limit users crazy_prof1,crazy_prof2 hosts {*} to h_vmem=10g
>  }
>
> This should have crazy_prof1 and crazy_prof2 limit 10G per slot on  
> any hosts
> Does this look right ? Make sense? Any downside to this?

This is only per slot, if it (h_vmem) is not consumable. Otherwise  
it's in total per node.

If you want it per slot, you could also define it in the queue  
configuration as h_vmem and disregard the RQS. But there it's unique  
for all users.

-- Reuti


> TIA
>
>
> On Sun, Nov 2, 2008 at 12:15 PM, Reuti <reuti at staff.uni-marburg.de>  
> wrote:
>> Am 02.11.2008 um 03:47 schrieb Mag Gam:
>>
>>> I have a job that creates 10 index jobs and executes.
>>>
>>> #!/bin/bash
>>> #$ -R y
>>> #$ -l h_vmem=16G
>>> #$ -t 1-10
>>>
>>> job -i file.$SGE_TASK_ID
>>>
>>> My question is since I will be running 10 sub jobs, would it reserve
>>> 16G each or 16G total?
>>
>> Array jobs are just multiple instances of the same job, all with  
>> the same
>> limits. So it will be per task. But it will not reserve anything,  
>> it's just
>> a limit per task.
>>
>>> I prefer we request for 16G each since this is
>>> a very memory intensive job. Also, is this the correct way to  
>>> reserve
>>> for resources?
>>
>> As it's a limit per task, it won't honor the already used memory  
>> on each
>> machine. Please have a look here, if you woukld like to have more  
>> advanced
>> memory handling:
>>
>> http://gridengine.sunsource.net/servlets/ReadMsg? 
>> list=users&msgNo=26701
>>
>> -- Reuti
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list