[GE users] SGE large memory jobs

m0zes adam.tygart at gmail.com
Fri Jul 17 19:50:51 BST 2009

    [ The following text is in the "utf-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

One thing to keep in mind, if you do this, it will make that memory
request to every host utilized by any job. So one job that uses
mem_free=2G, and two hosts, would request 2G of memory from both of
them, effectively requesting 4G which is more than necessary. This
could have some serious implications if your cluster is even
moderately memory hungry or have many jobs that utilize more than one

I hope that what I wrote was a comprehensible to you as it seems to me.


On Thu, Jul 16, 2009 at 15:10, matbradford<matthew.bradford at eds.com> wrote:
> If you put mem_free=<value> into the default sge_requests file, I think
> that should force it to be present in every submission request.
> Possibly.
> Cheers,
> Mat
>>-----Original Message-----
>>From: mbay2002 [mailto:jeff at haferman.com]
>>Sent: 16 July 2009 18:44
>>To: users at gridengine.sunsource.net
>>Subject: Re: [GE users] SGE large memory jobs
>>Thanks for the response.  My biggest concern now is that by making
>>mem_free a consumable, we need to make sure that all jobs request RAM.
>>I'll play around with this, I'm curious to see what happens if RAM is
>>not requested by the job submission once mem_free is made consumable...
>>adary wrote:
>>> Yes, you just need to change the NO to YES to make it consumable.
>>> For your other question, you need to add mem_free to each host, but
>>you can actually automate it a bit:
>>> for host in `qhost | sed '1,3d' | awk '{ print $1 }'`; do qconf
> -mattr
>>exechost complex_values mem_free=8G $host; done
>>> (if all your hosts are 8G hosts ofcourse)
>>> My recommendation is to always define hosts to 5% less ram than they
>>actually have, and to handle RAM requests accordingly, since some RAM
> is
>>always used by the OS itself
>>> As for consequences, once you make your mem_free consumable and your
>>jobs start requesting RAM, you need to make sure that all jobs that are
>>sent also request RAM.
>>> In my case we created a general wrapper for qsub that will always
>>request ram based on certain defaults, and each user can override it
>>with the need of the application.
> e
>>To unsubscribe from this discussion, e-mail: [users-
>>unsubscribe at gridengine.sunsource.net].
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=207608
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list