[GE users] h_vmem and mmap

reuti reuti at staff.uni-marburg.de
Tue May 25 19:26:46 BST 2010

Am 25.05.2010 um 19:59 schrieb reuti:

> Hi,
> Am 24.05.2010 um 16:31 schrieb fx:
>> murple <andreas.kuntzagk at mdc-berlin.de> writes:
>>> Hi,
>>> I just made h_vmem consumable to manage memory usage.
>>> Now one user askes how that's going to work out with his huge jobs using 
>>> mmap calls.
>>> He runs many different jobs in parallel (on one node) which access the 
>>> same big file (20G) with mmap calls.
>>> So the used/needed virtual memory for EACH job would look like 20G 
>>> meaning he can not run as many jobs in parallel as possible if the 
>>> "real" memory used would be taken into account.
>> I suspect you want a separate queue for such jobs/users that you know
>> are well-behaved, or maybe a resource quota.  I've been expecting to hit
>> this at some stage, but perhaps that's over-optimistic about how
>> sensibly applications are written.
> there was the RFE to have a new type of consumable: HOSTONCE - this will be subtracted once per host, even when it's used by different jobs. This could also cover this case.
> What about using an AR? During AR submisson, you request one time 20G (divided by slots) and a PE like SMP with the necessary number of slots for your serial jobs. For all the serial jobs which you submit into this AR, you don't request the memory again.

BTW: for now the reserved usage won't show up in `qhost -F`, but only in `qalter -w p <job_idf>` for waiting jobs or `qstat -j` w/o a <job_id> in case you enabled it with "schedd_job_info TRUE". It was already discussed on the list to make the output of `qhost -F` more explanatory.

> -- Reuti
>> (At least under Linux) h_data is documented not to include mmapped data,
>> but it's not a useful distinction from h_vmem since malloc typically
>> uses mmap, and I don't think there's a way to distinguish use of
>> anonymous mmaps.
>> -- 
>> Dave Love
>> Advanced Research Computing, Computing Services, University of Liverpool
>> AKA fx at gnu.org
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=258350
>> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=258501
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list