[GE users] [OT] How to know if it *really* uses InfiniBand ?
hristo at mc.phys.uni-sofia.bg
Mon May 31 10:55:57 BST 2010
MPI jobs first need to communicate various InfiniBand properties (like port GUIDs) before they could talk to each other. Also I/O redirection and other service information runs on top of TCP/IP.
Hope that helps,
On 21.05.2010, at 16:56, igardais wrote:
Our new toy is almost ready to use : we are using SGE over an ethernet connection and IB-RDMA for the MPI jobs.
We're using IntelMPI.
We set I_MPI_DEVICE=rdma and everything else to run over InfiniBand.
I_MPI_DEBUG=50 shows that the job selected "ofa-v2-ib0" for RDMA transfers, which is good.
But, when doing a 'netstat -tanpu', all the computing processes show they are connected using the ethernet IP address.
Is it OK or should I expect the processes to use the InfiniBand IP to confirm that they *really* uses InfiniBand for their chat ?
Excuse my newby questions but this look strange to me.
Dr Hristo Iliev
Monte Carlo research group
Faculty of Physics, University of Sofia
5 James Bourchier blvd, 1164 Sofia, Bulgaria
More information about the gridengine-users