testing 3 nodes at once

Question

I've been using the Infiniband commands (such as ib_send_bw, ibv_rc_pingpong, etc) to test latency and bandwidth between nodes (through the infiniband switch). However, I want to test more things. Is it possible to test latency and bandwith when two nodes send packets to a third node at the same time, or one node sends packets to two other nodes? If so, how?

Thanks

You can try any mpi implementation which gives an infiniband device layer support, latency difference wouldnt be noticable with mpi overhead if in C langauge. open-mpi, I prefer can help you, java mpi can have a little noticable overhead though — Aunn Raza, Oct 24 '14 at 08:44

score 0 · Accepted Answer · answered Jun 21 '14 at 06:41

Unfortunately, the libibverbs benchmarks are not sufficient for your case (for example, multiple nodes talking to one node). You need to write benchmarks of your own or look at existing code.

Take a look at the OSU benchmarks: http://mvapich.cse.ohio-state.edu/benchmarks/. I think some of them can measure many-to-many communication performance.

Here is a system (that I wrote) in which multiple client machines communicate with one server machine: https://github.com/anujkaliaiitd/HERD. In the default configuration, 36 client processes communicate with 6 server processes. The clients issue requests to the servers using RDMA writes over UC transport. The servers reply using SEND messages over UD transport.

I cannot guarantee that it will work for you. However, if you get bugs, I can offer some help.

testing 3 nodes at once

1 Answers1