[GE users] SGE large memory jobs

adary adary at marvell.com
Sun Jul 19 07:07:32 BST 2009


Its per slot unless you specify it by job ( sge 6.2 only )

When you define the consumable instead of YES write JOB and the consumable request will be per job and not per slot

-----Original Message-----
From: prentice [mailto:prentice at ias.edu]
Sent: Saturday, July 18, 2009 12:44 AM
To: users at gridengine.sunsource.net
Subject: Re: [GE users] SGE large memory jobs

mhanby wrote:
> I'm confused now :-) Is the mem_free consumed per slot, per compute node or per job?

It's per slot.

>
> Say the following job gets submitted to 4 compute node (each node has 8 cores and 24GB)
>
> qsub -pe openmpi 32 -l mem_free=2G
>
> Is this job consuming 64GB of RAM (2GB per slot), 8GB (2GB per compute node) or 2GB (2GB requested for the entire job, which would be 512MB per compute node)
>

This job is consuming 2 GB per slot, or 64 GB total.

Prentice
>
> -----Original Message-----
> From: m0zes [mailto:adam.tygart at gmail.com]
> Sent: Friday, July 17, 2009 1:51 PM
> To: users at gridengine.sunsource.net
> Subject: Re: [GE users] SGE large memory jobs
>
> One thing to keep in mind, if you do this, it will make that memory
> request to every host utilized by any job. So one job that uses
> mem_free=2G, and two hosts, would request 2G of memory from both of
> them, effectively requesting 4G which is more than necessary. This
> could have some serious implications if your cluster is even
> moderately memory hungry or have many jobs that utilize more than one
> host.
>
> I hope that what I wrote was a comprehensible to you as it seems to me.
>
> --
> Adam
>
> On Thu, Jul 16, 2009 at 15:10, matbradford<matthew.bradford at eds.com> wrote:
>> If you put mem_free=<value> into the default sge_requests file, I think
>> that should force it to be present in every submission request.
>> Possibly.
>>
>> Cheers,
>>
>> Mat
>>
>>> -----Original Message-----
>>> From: mbay2002 [mailto:jeff at haferman.com]
>>> Sent: 16 July 2009 18:44
>>> To: users at gridengine.sunsource.net
>>> Subject: Re: [GE users] SGE large memory jobs
>>>
>>> Thanks for the response.  My biggest concern now is that by making
>>> mem_free a consumable, we need to make sure that all jobs request RAM.
>>>
>>> I'll play around with this, I'm curious to see what happens if RAM is
>>> not requested by the job submission once mem_free is made consumable...
>>>
>>>
>>>
>>> adary wrote:
>>>> Yes, you just need to change the NO to YES to make it consumable.
>>>>
>>>> For your other question, you need to add mem_free to each host, but
>>> you can actually automate it a bit:
>>>> for host in `qhost | sed '1,3d' | awk '{ print $1 }'`; do qconf
>> -mattr
>>> exechost complex_values mem_free=8G $host; done
>>>> (if all your hosts are 8G hosts ofcourse)
>>>>
>>>> My recommendation is to always define hosts to 5% less ram than they
>>> actually have, and to handle RAM requests accordingly, since some RAM
>> is
>>> always used by the OS itself
>>>> As for consequences, once you make your mem_free consumable and your
>>> jobs start requesting RAM, you need to make sure that all jobs that are
>>> sent also request RAM.
>>>> In my case we created a general wrapper for qsub that will always
>>> request ram based on certain defaults, and each user can override it
>>> with the need of the application.
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessag
>> e
>>> Id=207582
>>>
>>> To unsubscribe from this discussion, e-mail: [users-
>>> unsubscribe at gridengine.sunsource.net].
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=207608
>>
>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>>
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=207844
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=207860
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=207861

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=208162

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list