[GE users] h_vmem and mmap

fx d.love at liverpool.ac.uk
Mon May 24 15:31:36 BST 2010


murple <andreas.kuntzagk at mdc-berlin.de> writes:

> Hi,
>
> I just made h_vmem consumable to manage memory usage.
> Now one user askes how that's going to work out with his huge jobs using 
> mmap calls.
>
> He runs many different jobs in parallel (on one node) which access the 
> same big file (20G) with mmap calls.
> So the used/needed virtual memory for EACH job would look like 20G 
> meaning he can not run as many jobs in parallel as possible if the 
> "real" memory used would be taken into account.

I suspect you want a separate queue for such jobs/users that you know
are well-behaved, or maybe a resource quota.  I've been expecting to hit
this at some stage, but perhaps that's over-optimistic about how
sensibly applications are written.

(At least under Linux) h_data is documented not to include mmapped data,
but it's not a useful distinction from h_vmem since malloc typically
uses mmap, and I don't think there's a way to distinguish use of
anonymous mmaps.

-- 
Dave Love
Advanced Research Computing, Computing Services, University of Liverpool
AKA fx at gnu.org

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=258350

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list