[GE users] MPICH h_vmem questions

Reuti reuti at staff.uni-marburg.de
Wed Aug 2 16:03:58 BST 2006


Am 02.08.2006 um 15:17 schrieb Dev:

> Hi,
>
>       Few questions about h_vmem and MPICH.
>
> Setup
>
> Two Nodes with h_vmem set to requestable and consumable and with  
> its value
> 6.519G when doing a qhost -F | grep h_vmem
>
> Both the nodes have 4 slots each.
>
>
> I have a very simple MPICH program which doesn't do anything useful  
> except mallocing memory in each MPICH process by taking a command  
> line parameter.
>
> Each MPICH process mallocs different amounts of memory based on a  
> command line parameter specified by the user.
>
> I start the MPI program by doing a qsub and requesting for h_vmem  
> and a parallel environment which includes both the nodes mentioned  
> above.
>
> The whole purpose is just to increase my understanding of how  
> h_vmem behaves when launching mpich programs and I'm not yet an  
> expert MPICH programmer.
>
>
>
> I do
>
> qsub -pe mp_pe 8 -l h_vmem=1g submit_mympi.sh
>
> In submit_mympi.sh I ask the MPI program to allocate a total of  
> less than 6G approx( more closer to 6.5 or so )  and everything  
> works fine with the mpich processes distributed between both the  
> nodes.
>
> Then I do
>
> qsub -pe mp_pe 5 -l h_vmem=2g submit_mympi.sh

Which rank is allocating how much memory? Is it proper distributed?  
Are you using MPICH with qrsh or shared memory with forks? - Reuti


> In submit_mympi.sh I ask the MPI program to allocate a total of  
> 6.5G approx and the MPICH processes gets killed.
>
> What I expected in the above case was for the program to run since  
> the total amount of h_vmem requested was more than what  the  
> program tried allocating with 4 mpich processes on each node. 
> ( Definitely the total mallocing attempted on a single node would  
> have been less than 6.5 GB which is the total setting for h_vmem on  
> each node ).
>
> Is this the intended behavior that, even if the total h_vmem  
> requested is more than the total memory which would be used by the  
> job , the job will get killed if the memory allocated is more than  
> the value of h_vmem on the head node?
>
>
> cheers
>
> /Dev
>
>
>
>
>
>
>
>
>
>
> Do you Yahoo!?
> Get on board. You're invited to try the new Yahoo! Mail Beta.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list