[GE users] ram limiting users per node

danielgoolsby danielgoolsby at gmail.com
Thu Mar 4 21:50:15 GMT 2010


Finally found my problem.  I was using qmon to set the consumable.  It
saved it when I would hit modify, then commit, but it would never show
up as a complex when I went to edit a queue or a host.

I finally tried via command-line, and it worked as intended.

Thanks for the help.

--daniel



On 3/3/10, reuti <reuti at staff.uni-marburg.de> wrote:
>> -----Original Message-----
>> From: reuti [mailto:reuti at staff.uni-marburg.de]
>> Sent: Wednesday, March 03, 2010 2:41 AM
>> To: users at gridengine.sunsource.net
>> Subject: Re: [GE users] ram limiting users per node
>>
>> Am 03.03.2010 um 03:10 schrieb danielgoolsby:
>>
>>> I see where you can check whether or not there's enough free space..
>>> but how can you tell the scheduler that your process (or slot) will
>>> need up to 7 gigs of ram.  Half of my machines have 24 gigs of ram
>>> which could only run three of these jobs, but the other half has 48
>>> gigs of ram which could run 7 of the jobs.  How would the scheduler
>>> know that my process would need at most 7 gigs.. then when it went
>>> to allocate the jobs, not to exceed the upper hard limit of ram that
>>> i'm requesting per slot?
>>
>> Exactly this is covered in the link I posted, ...
>>
>>
>>> Is there a way with qsub to specify that you'll need 7 gigs of
>>> memory, and only run if there's at least 8 gigs of mem_free (-l mf).
>>
>> not to use mem_free but h_vmem or virtual_free as a consumable.
>>
>> -- Reuti
>>
>>
>>>  I understand the mem_free part, but is there a way to do the
>>> 'allocate 7 gigs of memory' up front?
>>>
>>> That make any sense?
>>>
>>> Daniel
>>>
>>> On Tue, Mar 2, 2010 at 6:17 AM, reuti <reuti at staff.uni-marburg.de>
>>> wrote:
>>> Hi,
>>>
>>> Am 02.03.2010 um 12:57 schrieb danielgoolsby:
>>>
>>>> I have about 40 dual quad core, half have 24 gigs of ram, other
>>>> half has 48 gigs of ram a piece.  most users jobs can fill up 8
>>>> slots per node easily and not use much ram, but I do have a few
>>>> users that have some ram intensive matlab jobs that take up about
>>>> 6-8 gigs of ram per job.
>>>>
>>>> obviously if someone submitted enough jobs (in the grid's current
>>>> config), they'll put 8 slots on one machine and you'd kill the
>>>> machine due to ram limitation.
>>>>
>>>> what i'm curious is how do you specify max ram usage per user per
>>>> node?  if the job gets initially submitted and says it uses 1 gig
>>>> of ram, and over time it grows to 8 gigs,  if there was a
>>>> limitation in place, what happens?  does all 8 slots fill up, then
>>>> if the user violates the ram contraints, does it just suspend jobs
>>>> until some other processes finishes?
>>>
>>> a usual approach is:
>>>
>>> http://gridengine.info/2009/12/01/adding-memory-requirement-
>>> awareness-
>>> to-the-scheduler
>>>
>>> You could even limit the maximum per user per node with an RQS, but I
>>> think this is not necessary at the first place.Am 03.03.2010 um
>>> 15:47 schrieb mhanby:
>
>
>> For this to work, don't all jobs have to request h_vmem or
>> virtual_free (or have a default if not requested)? Otherwise the
>> other jobs running on the node can run wild.
>
> In the complex configuration a default can be given which will be
> used then. In fact: if you specify 0 MB for h_vmem, the job will
> never start due to this contraint.
>
> -- Reuti
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=246873
>
> To unsubscribe from this discussion, e-mail:
> [users-unsubscribe at gridengine.sunsource.net].
>


-- 
--daniel
--

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=247118

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list