[GE users] limiting the physical memory and virtual memory usage on a host

Reuti reuti at staff.uni-marburg.de
Wed Mar 8 19:52:35 GMT 2006


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Jinal,

Quoting Jinal Jhaveri <jajhaveri at lbl.gov>:

> Thanks Reuti,
>
> I would like to avoid the user to request how much they need because 
> that will require change in the scripts they have written. So if I 
> execute the following command:
>
> qconf -mattr exechost complex_values mem_total=3.5G  `qselect`
> and qconf -mattr exechost complex_values virtual_total=5G `qselect`
>
> will it do it?

no - this will just set the amount of this available resource. If you have a
mixture of jobs with a high memory demand, and also jobs with less
requirements, they must request something to allow SGE to make the best
scheduling, and avoid two memory intensive jobs on one node.

>
> The total physical memory on my system is 4G and swap space is 5.3
>
>
>
> OR you think that making the complex virtual_free consumable and 
> requestable and default it to 2G or something (though I don't want

If you can live with 2 GB for all jobs, then you could just set h_vmem in the
queue definition to 2G. This will be then a hard limit for these jobs of
course. Then you don't need a consumable or default request.

-- Reuti


> users to have to change their scripts and specifically ask for that 
> much free vmem). The reason I am asking all these questions is that,
> sometimes we have issues where multiple memory intensive jobs are 
> scheduled to the same node and due to that either they are thrashing 
> or some of them seg fault putting the node in error state. I would 
> like to avoid that situation.
>
> Thanks
> --Jinal
>
>
>
>
> Reuti wrote:
>
>> Jinal,
>>
>> similar discussions were on this list before:
>>
>> http://gridengine.sunsource.net/servlets/ReadMsg?listName=users&msgNo=10553 
>> In addition you could make h_vmem consumable and give it also an 
>> inital value.
>> Jobs have of course to request both h_vmem and virtual_free in this case. I
>> think, it should be sufficient to use just one of them: h_vmem for enforced
>> limits, virtual_free for fair users.
>>
>> HTH - Reuti
>>
>>
>> Quoting Jinal Jhaveri <jajhaveri at lbl.gov>:
>>
>>> Hi All,
>>>
>>> I would like to limit the total amount of physical memory as well 
>>> as virtual memory, all the jobs collectively use on an exechost. 
>>> Any suggestions on how to do that? I know that I can't change the 
>>> load_values, so I am pretty sure there should be some way of doing 
>>> it either via complex or something else.
>>>
>>> thank you very much for your help.
>>>
>>> --Jinal
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list