[GE users] Large memory consumption of qrsh using builtin method

jlopez jlopez at cesga.es
Fri Feb 27 10:03:15 GMT 2009


Hi,

We  have found the reason of the large memory consumption of qrsh using 
builtin method in IA64. It is all due to the fact that we are using a 
stack limit of 256MB.

These are the facts conclusions that I think could be helpful to others 
experiencing similar problems:
- qrsh uses threads: 3 in general and 4 if it has to expand the builtin 
shell (this can be avoided using the -noshell option). Bash is not 
recognised as a supported shell so if the rsh wrapper is written using 
bash qrsh would expand the builtin shell unless the -noshell option is 
given.
- Memory consumption is about two times the value of stack limit 
established in the system (ulimit -s). In case the stack is unlimited it 
seems that a reference value of 32MB is used in IA64.
- Optimum memory consumption is obtained with a stack value of 1MB 
(ulimit -s 1024). In this case qrsh consumes 4MB if no builtin shell is 
expanded or 10MB if the shell is expanded.

The final conclusion is that the best alternative in IA64 is to set the 
stack limit to 1024 and write the rsh wrapper using sh.

I hope this can be useful to others experiencing similar problems.

Cheers,
Javier

jlopez wrote:
> jlopez wrote:
>   
>>>> We are using it on IA64 (Itanium Montvale) under Linux.
>>>>
>>>> Could it be a problem in the IA64 version of qrsh?
>>>>     
>>>>       
>>>>         
>>> Javier,
>>>
>>> I don't have access to an IA64 machine. Maybe someone else could post  
>>> his findings about this issue?
>>>
>>> -- Reuti
>>>
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=103502
>>>
>>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>>>   
>>>     
>>>       
>> Thanks Reuti,
>>
>> Let's see if other people with IA64 see the same behavior. In the 
>> meanwhile I will do some tests to see if the memory consumption is 
>> related to the builtin communication mechanism or inherit to the qrsh 
>> version in 6.2 (in 6.1 we do not see this).
>>
>> Cheers,
>> Javier
>>
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=103872
>>
>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>>     
> There must be a memory leak in IA64 because there are qrsh processes 
> that are consuming 4GB of virtual memory after several hours running:
> 25141 csedamsp  15   0 4104m 3984 3296 S    0  0.0   0:00.00 
> qrsh                                                                           
>
> 25142 csedamsp  16   0 4104m 3984 3296 S    0  0.0   0:00.01 
> qrsh                                                                           
>
> 25140 csedamsp  15   0 4103m 3968 3296 S    0  0.0   0:00.01 
> qrsh                                                                           
>
> 25143 csedamsp  15   0 4103m 3968 3296 S    0  0.0   0:00.02 qrsh
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=104185
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=115973

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

    [ Part 2, "jlopez.vcf"  Text/X-VCARD (Name: "jlopez.vcf") ~367 bytes. ]
    [ Unable to print this part. ]



More information about the gridengine-users mailing list