[GE users] Memory hard limit seems to be ignored on sge 5.3p5 version

Shannon V. Davidson svdavidson at swbell.net
Mon Sep 13 22:51:57 BST 2004


    [ The following text is in the "windows-1252" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

I'm not sure why your job is not being killed, but keep in mind that 
h_data gets set as a process limit.  If you have multiple processes in 
your job and each process is under the limit, your job can still use 
more 1.5GB of memory.  You might try using h_vmem, instead.  This is 
implemented by Grid Engine as a job limit.  The execution daemon will 
kill the job if the sum of the virtual memory being used by the 
processes in the job exceeds the specified h_vmem limit.  For more 
details about job limits, take a look at the queue_conf(5) man page 
under RESOURCE LIMITS.  BTW, using h_vmem will only work on 
architectures upon which the execution daemon dynamically collects 
process usage, such as Linux, FreeBSD, Solaris, IRIX, UNICOS, Darwin, 
and HP-UX.

Shannon


Korambath, Prakashan wrote:

>Thanks Reuti.  Now the output looks like below.  The running job which uses 1.9 GB is still continuing on the compute nodes.  
>
>Prakashan
>
>
>
>core file size        (blocks, -c) 1536000
>data seg size         (kbytes, -d) 1536000
>file size             (blocks, -f) unlimited
>max locked memory     (kbytes, -l) unlimited
>max memory size       (kbytes, -m) unlimited
>open files                    (-n) 1024
>pipe size          (512 bytes, -p) 8
>stack size            (kbytes, -s) unlimited
>cpu time             (seconds, -t) unlimited
>max user processes            (-u) 7168
>virtual memory        (kbytes, -v) unlimited
>
>core file size        (blocks, -c) 1536000
>data seg size         (kbytes, -d) 1536000
>file size             (blocks, -f) unlimited
>max locked memory     (kbytes, -l) unlimited
>max memory size       (kbytes, -m) unlimited
>open files                    (-n) 1024
>pipe size          (512 bytes, -p) 8
>stack size            (kbytes, -s) unlimited
>cpu time             (seconds, -t) unlimited
>max user processes            (-u) 7168
>virtual memory        (kbytes, -v) unlimited
>
>output of qconf -sq s04H command
>
>s_rt                 INFINITY
>h_rt                 INFINITY
>s_cpu                INFINITY
>h_cpu                INFINITY
>s_fsize              INFINITY
>h_fsize              INFINITY
>s_data               1500M
>h_data               1500M
>s_stack              INFINITY
>h_stack              INFINITY
>s_core               INFINITY
>h_core               1500M
>s_rss                INFINITY
>h_rss                INFINITY
>s_vmem               INFINITY
>h_vmem               INFINITY
>
>Prakashan
>
>
>-----Original Message-----
>From:	Reuti [mailto:reuti at staff.uni-marburg.de]
>Sent:	Mon 9/13/2004 11:27 AM
>To:	users at gridengine.sunsource.net
>Cc:	
>Subject:	RE: [GE users] Memory hard limit seems to be ignored on sge 5.3p5 version 
>Please try 1500M in the queue definition instead of G. - Reuti
>
>Quoting "Korambath, Prakashan" <ppk at ats.ucla.edu>:
>
>  
>
>>Below is the output of what I am getting from the script
>>#!/bin/sh
>>ulimit -aH
>>echo
>>ulimit -aS
>>
>>---------------------
>>
>>core file size        (blocks, -c) 1048576
>>data seg size         (kbytes, -d) 1048576
>>file size             (blocks, -f) unlimited
>>max locked memory     (kbytes, -l) unlimited
>>max memory size       (kbytes, -m) unlimited
>>open files                    (-n) 1024
>>pipe size          (512 bytes, -p) 8
>>stack size            (kbytes, -s) unlimited
>>cpu time             (seconds, -t) unlimited
>>max user processes            (-u) 7168
>>virtual memory        (kbytes, -v) unlimited
>>
>>core file size        (blocks, -c) 1048576
>>data seg size         (kbytes, -d) 1048576
>>file size             (blocks, -f) unlimited
>>max locked memory     (kbytes, -l) unlimited
>>max memory size       (kbytes, -m) unlimited
>>open files                    (-n) 1024
>>pipe size          (512 bytes, -p) 8
>>stack size            (kbytes, -s) unlimited
>>cpu time             (seconds, -t) unlimited
>>max user processes            (-u) 7168
>>virtual memory        (kbytes, -v) unlimited
>>
>>ouput of the  command qconf -sq s04H on the node where it ran.
>>
>>s_rt                 INFINITY
>>h_rt                 INFINITY
>>s_cpu                INFINITY
>>h_cpu                INFINITY
>>s_fsize              INFINITY
>>h_fsize              INFINITY
>>s_data               1.5G
>>h_data               1.5G
>>s_stack              INFINITY
>>h_stack              INFINITY
>>s_core               INFINITY
>>h_core               1.5G
>>s_rss                INFINITY
>>h_rss                INFINITY
>>s_vmem               INFINITY
>>h_vmem               INFINITY
>>
>>
>>Prakashan
>>
>>
>>-----Original Message-----
>>From:	Ron Chen [mailto:ron_chen_123 at yahoo.com]
>>Sent:	Mon 9/13/2004 10:43 AM
>>To:	users at gridengine.sunsource.net
>>Cc:	
>>Subject:	RE: [GE users] Memory hard limit seems to be ignored on sge 5.3p5
>>version 
>>Can you run the job that prints the resource limits
>>again with the new setting?
>>
>> -Ron
>>
>>--- "Korambath, Prakashan" <ppk at ats.ucla.edu> wrote:
>>    
>>
>>>I just set the soft limit s_data to 1.5G as well.  
>>>
>>>
>>>s_rt                 INFINITY
>>>h_rt                 INFINITY
>>>s_cpu                INFINITY
>>>h_cpu                INFINITY
>>>s_fsize              INFINITY
>>>h_fsize              1.5G
>>>s_data               1.5G
>>>h_data               1.5G
>>>s_stack              INFINITY
>>>h_stack              INFINITY
>>>s_core               1.5G
>>>h_core               1.5G
>>>s_rss                INFINITY
>>>h_rss                INFINITY
>>>s_vmem               INFINITY
>>>h_vmem               INFINITY
>>>
>>>Prakashan
>>>
>>>
>>>-----Original Message-----
>>>From:	Rayson Ho [mailto:raysonho at eseenet.com]
>>>Sent:	Mon 9/13/2004 11:04 AM
>>>To:	users at gridengine.sunsource.net
>>>Cc:	
>>>Subject:	RE: [GE users] Memory hard limit seems to
>>>be ignored on sge 5.3p5 version 
>>>You have:
>>>
>>>s_data               INFINITY
>>>h_data               1.5G
>>>
>>>However, in the shepherd code:
>>>
>>> /* hard limit must be greater or equal to soft
>>>limit */
>>> if (rlp->rlim_max < rlp->rlim_cur)
>>>    rlp->rlim_cur = rlp->rlim_max;
>>>
>>>Can you set the s_data to 1.5G instead of
>>>"INFINITY"??
>>>
>>>Rayson
>>>
>>>
>>>
>>>      
>>>
>>>>Thanks. The cluster is running Red Hat Linux 8.0
>>>>        
>>>>
>>>with 
>>>2.4.20-30.8.legacysmp kernel.  The output of the
>>>limit command is as
>>>follows
>>>      
>>>
>>>>cputime         unlimited
>>>>filesize        1048576 kbytes
>>>>datasize        1048576 kbytes
>>>>stacksize       unlimited
>>>>coredumpsize    1048576 kbytes
>>>>memoryuse       unlimited
>>>>vmemoryuse      unlimited
>>>>descriptors     1024
>>>>memorylocked    unlimited
>>>>maxproc         7168
>>>>        
>>>>
>>---------------------------------------------------------
>>    
>>
>>>Get your FREE E-mail account at
>>>http://www.eseenet.com !
>>>
>>>
>>>      
>>>
>>---------------------------------------------------------------------
>>    
>>
>>>To unsubscribe, e-mail:
>>>users-unsubscribe at gridengine.sunsource.net
>>>For additional commands, e-mail:
>>>users-help at gridengine.sunsource.net
>>>
>>>
>>>
>>>
>>>
>>>
>>>      
>>>
>>---------------------------------------------------------------------
>>    
>>
>>>To unsubscribe, e-mail:
>>>users-unsubscribe at gridengine.sunsource.net
>>>For additional commands, e-mail:
>>>users-help at gridengine.sunsource.net
>>>
>>>
>>>      
>>>
>>
>>		
>>__________________________________
>>Do you Yahoo!?
>>Yahoo! Mail - 50x more storage than other providers!
>>http://promotions.yahoo.com/new_mail
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
>>
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
>>    
>>
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>
>  
>


-- 
___________________________________________

Shannon V. Davidson <svdavidson at swbell.net>
Senior Software Engineer           Raytheon
636-479-7465 office        443-383-0331 fax
___________________________________________





More information about the gridengine-users mailing list