[GE users] Is there a way to reserve the memory for the process inste

Reuti reuti at staff.uni-marburg.de
Thu Apr 10 10:41:21 BST 2008


Hi,

Am 09.04.2008 um 17:50 schrieb Mulley, Nikhil:
> SGE is v6.0.u11. One of the users had asked us recently if the memory
> could be reserved by a process run via sge, that the memory may not be
> used by the process but is allocated to in view of the sge. ?
> This has put us to think of advanced reservations, but given the  
> version
> we are running we were not sure if there is anything such like  
> could be
> done.
> We actually were so far suggesting to use 'mem_free' resource  
> attribute
> that would ensure that the machines has enough memory before it gets
> launched, but might not immediately be taking the mentioned memory.
> Sometimes there is another process which gets scheduled on the same
> machine getting smitten by the mem_free attribute and might take up  
> the
> memory immediately what it has mentioned.
> So, Is there a way to reserve the memory, instead of just checking for
> free memory?
>
> He has even come up with this link where there is 'mem_token' resource
> attribute is spoken about, but I am sure at the moment on how to go
> about it and what does it take me to get this complex added.
>
> http://www.lancs.ac.uk/iss/hpc/advanced_jobs.html


you have two options:

a) A hard limit by making h_vmem "consumable" in the complex  
configuration with a proper default request and assigning the  
installed physical RAM per node to this complex. If the job exceeds  
this limit which it requests by e.g. "-l h_vmem=1G", they will be  
killed.

b) Make virtual_free consumable in the complex configuration with a  
proper default request and assigning the installed physical RAM per  
node to this complex. The available amount of virtual_free is the  
lower value of 1) SGE's simple subtraction of the used up  
virtual_free and 2) the measured virtual_free on the node.

Difference is just that a) will kill the jobs if they consume more  
memory, while b) is only a guidance for SGE and it's the  
responsibility of the user not to abuse this feature. With our long  
running jobs we prefer b), as otherwise one byte too much might kill  
the job and we would lose much computing time.

-- Reuti

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list