-1

How can I collect an MPI communication trace on Supercomputers?

I need text files with details of each message (say sender, receiver, size, etc.) that I can parse. I was using following command for Intel MPI and do not see any text files.

mpirun -trace -n 4 -trace-pt2pt -trace-collectives ./myApp
Ronny Brendel
  • 4,777
  • 5
  • 35
  • 55
SummonersRift
  • 51
  • 1
  • 1
  • 7

1 Answers1

3

I am not familiar with Intel MPI's integrated solution.

There is a number of tools that provide MPI tracing.

Performance focussed:

Correctness checking:

I recommend to not roll your own solution, because it's not straight forward to match receives to sends and you might run into timing issues because timers are not synchronized across nodes.

You could e.g. trace a run using Score-P, and then use the otf2-print command on the trace to get the text output you wanted. Or you can use the OTF2 reader library and develop a tool on top of it. Here is a short tutorial on how to run Score-P, starting at slide 17

Ronny Brendel
  • 4,777
  • 5
  • 35
  • 55
  • Thanks for the reply. I was able to use mpirun -trace -trace-pt2pt for collecting the traces using Intel MPI. The trace is output in .stf format which can be analyzed with Intel trace analyzer. – SummonersRift Jan 17 '18 at 20:55