[GE users] threaded jobs (no PE) and consumable memory

reuti reuti at staff.uni-marburg.de
Thu Jan 14 20:16:04 GMT 2010


Am 14.01.2010 um 17:05 schrieb txema_heredia:

>> Am 14.01.2010 um 15:25 schrieb txema_heredia:
>>
>>> Thanks for your answers
>>>
>>>> Hi,
>>>>
>>>> you might try to set an additional "-l h_stack=20M" (or whatever  
>>>> size
>>>> you need), as some threaded applications allocate the whole stack
>>>> space
>>>> (which is in older versions of the SGE set equally to the h_vmem
>>>> size -
>>>> in newer versions it is left as "unlimited", therefore avoiding  
>>>> this
>>>> problem).
>>>>
>>>> Hope it helps,
>>>> Sabine
>>>
>>> I have tried it alone (-l h_stack=4G), and the program runs OK. The
>>> problem is that this attribute is not restrictive at all, so I have
>>> used -l h_stack=50M, but the job is still able to use 4G without
>>> receiving any termination signal (unlike if I use -l h_vmem=50M,
>>> which aborts the job when exceeded).
>>>
>>> If combined -l h_stack and -l h_vmem, h_vmem takes preference and
>>> the job is killed as usual (the malloc error and segment violation
>>> thing).
>>
>> You also tried it with different settings: -l
>> h_stack=50M,h_vmem=4G ... ?
>>
>> -- Reuti
>>
>
> Thank you very much Sabine and Reuti, that worked! At first I  
> didn't understood Sabine's answer, but combining both h_stack and  
> h_vmem, it works. In fact, the crashing malloc is now asking for  
> the h_stack memory amount.
>
> I've tried h_stack=50M and it worked fine. On the other hand,  
> h_stack=1G worked but really slow, and in the previous set of  
> tests, without any memory-related option, I saw malloc is  
> allocating ~20M.
>
> So this arises a new question: Is there any way or best practice to  
> determine how big my stack should be?

Exactly what you did: your experience will tell you. But values  
between 32M and 128M seem to be fine for most applications.

-- Reuti


>
> Txema
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=238781
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=238828

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list