[GE users] SGE large memory jobs

mhanby mhanby at uab.edu
Fri Jul 17 19:09:43 BST 2009


Next you get to decide what a good default value is for your environment. You might take a look at the qhost output to while your normal load of jobs are running. This will give you a snapshot in time of memory usage per node.

Mike

-----Original Message-----
From: matbradford [mailto:matthew.bradford at eds.com] 
Sent: Thursday, July 16, 2009 3:11 PM
To: users at gridengine.sunsource.net
Subject: RE: [GE users] SGE large memory jobs

If you put mem_free=<value> into the default sge_requests file, I think
that should force it to be present in every submission request.
Possibly.

Cheers,

Mat

>-----Original Message-----
>From: mbay2002 [mailto:jeff at haferman.com]
>Sent: 16 July 2009 18:44
>To: users at gridengine.sunsource.net
>Subject: Re: [GE users] SGE large memory jobs
>
>Thanks for the response.  My biggest concern now is that by making
>mem_free a consumable, we need to make sure that all jobs request RAM.
>
>I'll play around with this, I'm curious to see what happens if RAM is
>not requested by the job submission once mem_free is made consumable...
>
>
>
>adary wrote:
>> Yes, you just need to change the NO to YES to make it consumable.
>>
>> For your other question, you need to add mem_free to each host, but
>you can actually automate it a bit:
>>
>> for host in `qhost | sed '1,3d' | awk '{ print $1 }'`; do qconf
-mattr
>exechost complex_values mem_free=8G $host; done
>>
>> (if all your hosts are 8G hosts ofcourse)
>>
>> My recommendation is to always define hosts to 5% less ram than they
>actually have, and to handle RAM requests accordingly, since some RAM
is
>always used by the OS itself
>>
>> As for consequences, once you make your mem_free consumable and your
>jobs start requesting RAM, you need to make sure that all jobs that are
>sent also request RAM.
>>
>> In my case we created a general wrapper for qsub that will always
>request ram based on certain defaults, and each user can override it
>with the need of the application.
>>
>
>------------------------------------------------------
>http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessag
e
>Id=207582
>
>To unsubscribe from this discussion, e-mail: [users-
>unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=207608

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=207836

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list