[GE users] MPICH h_vmem questions

Dev dev_hyd2001 at yahoo.com
Wed Aug 2 14:17:56 BST 2006

    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]


      Few questions about h_vmem and MPICH.


Two Nodes with h_vmem set to requestable and consumable and with its value
6.519G when doing a qhost -F | grep h_vmem

Both the nodes have 4 slots each.

I have a very simple MPICH program which doesn't do anything useful except mallocing memory in each MPICH process by taking a command line parameter.

Each MPICH process mallocs different amounts of memory based on a command line parameter specified by the user.

I start the MPI program by doing a qsub and requesting for h_vmem and a parallel environment which includes both the nodes mentioned above.

The whole purpose is just to increase my understanding of how h_vmem behaves when launching mpich programs and I'm not yet an expert MPICH programmer.

I do 

qsub -pe mp_pe 8 -l h_vmem=1g submit_mympi.sh

In submit_mympi.sh I ask the MPI program to allocate a total of less than 6G approx( more closer to 6.5 or so )  and everything works fine with the mpich processes distributed between both the nodes.

Then I do 

qsub -pe mp_pe 5 -l h_vmem=2g submit_mympi.sh

In submit_mympi.sh I ask the MPI program to allocate a total of 6.5G approx and the MPICH processes gets killed.

What I expected in the above case was for the program to run since the total amount of h_vmem requested was more than what  the program tried allocating with 4 mpich processes on each node.( Definitely the total mallocing attempted on a single node would have been less than 6.5 GB which is the total setting for h_vmem on each node ).

Is this the intended behavior that, even if the total h_vmem requested is more than the total memory which would be used by the job , the job will get killed if the memory allocated is more than the value of h_vmem on the head node?



Do you Yahoo!?
 Get on board. You're invited to try the new Yahoo! Mail Beta.

More information about the gridengine-users mailing list