[GE users] Ideas requested regarding jobs and memory consumption

reuti reuti at staff.uni-marburg.de
Fri Feb 6 11:48:21 GMT 2009


Hiho,

Am 06.02.2009 um 00:59 schrieb maritac:

> I have an issue that I'm trying to help one of my SGE users  
> resolve, and
> I'd appreciate any ideas that the user group might have.
>
> When submitting a job with a requirement for a specific amount of free
> memory, it takes some time after the job lands on the exec host for  
> the
> job to consume memory.  This means that for a period of time the exec
> host will report more memory available than will be available once the
> job gets fully running.  In the meantime more jobs can land on the
> system because the mem_free is reporting available memory, thus
> consuming too much memory by the time all the jobs get to full speed.
>
> My user needs to find a way to prevent additional jobs from landing on
> the exec host while the initial job gets up to full memory usage. What
> makes this especially difficult is that making configuration  
> changes to
> the grid isn't an option, as this has to work out-of-the-box at our
> customers' sites.  There is no way of predicting how the customer's  
> grid
> might be configured, and we can't ask them to make configuration
> changes.
>
> One idea being considered is slowing down the job submission to allow
> the initial job more time to consume its full amount of required  
> memory
> before the next batch of jobs is submitted.  Anyone have a similar
> situation, or any additional ideas?

if you slow down the job submission, you gain nothing. How should SGE  
know, that a submitted job will need in the end 2 GB - there is no  
crystal ball feature built into SGE. So the users must specify their  
requirements while they submit the job. And if they do so, they also  
setup something in the cluster to handle these resource requests -  
which can of course be done in various ways.

-- Reuti


> Thanks for your help.
> Marita
>
> ----------------------------
> Marita Christiansen
> Grid Support
> Engineering Tools and Architecture
> Mentor Graphics Corporation
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=102273
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=102377

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list