[GE users] [OT] How to know if it *really* uses InfiniBand ?
ppk at ats.ucla.edu
Mon May 31 18:54:45 BST 2010
We normally run these codes
http://mvapich.cse.ohio-state.edu/benchmarks/. If your
latency between two nodes is close to 4 microseconds, you
are pretty much sure you are using IB hardware.
> MPI jobs first need to communicate various InfiniBand properties (like port GUIDs) before they could talk to each other. Also I/O redirection and other service information runs on top of TCP/IP.
> Hope that helps,
> On 21.05.2010, at 16:56, igardais wrote:
> Hi list,
> Our new toy is almost ready to use : we are using SGE over an ethernet connection and IB-RDMA for the MPI jobs.
> We're using IntelMPI.
> We set I_MPI_DEVICE=rdma and everything else to run over InfiniBand.
> I_MPI_DEBUG=50 shows that the job selected "ofa-v2-ib0" for RDMA transfers, which is good.
> But, when doing a 'netstat -tanpu', all the computing processes show they are connected using the ethernet IP address.
> Is it OK or should I expect the processes to use the InfiniBand IP to confirm that they *really* uses InfiniBand for their chat ?
> Excuse my newby questions but this look strange to me.
> Dr Hristo Iliev
> Monte Carlo research group
> Faculty of Physics, University of Sofia
> 5 James Bourchier blvd, 1164 Sofia, Bulgaria
To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
More information about the gridengine-users