[GE users] couldn't create pipe: too many open files

reuti reuti at staff.uni-marburg.de
Tue Nov 17 09:25:22 GMT 2009


Am 16.11.2009 um 18:30 schrieb richmaes:

> I think this is a linux issue and not so much a SGE issue, but I  
> figured that someone must have hit this error before.
> The exact text of the error is
> couldn't create pipe: too many open files
>
> This happens about after submission of 700+ jobs into the queue.
>
> I can go to another shell and submit another 700 jobs without issue
>
> I think what has happened is that there is shell based limit or  
> maybe a PID based limit that is currently preventing more than 16K  
> open file handles per shell or PID.
>
> From /proc/sys/fs/file-max I have 371334 set.
>
> So I don't don't know why it's choking at 16K file open files.
>
> BTW, all my jobs are launched from a third party script that we  
> didn't write.  But ideally, it should be able to submit 1000+ jobs.
>
> Has anyone dealt with this before?

Maybe these are the limits of the shell. What is "open files" set to in:

ulimit -aH
ulimit -aS

which you could adjust. Your application is keeping these processes  
open during its lifetime?

-- Reuti


> Rich
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=227275
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=227416

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list