2

Suppose I have two process A and B. A is sending some numbers to B. But B doesn't know how many. B is using MPI_Probe to probe about the number of items and then allocates a buffer to receive those numbers.

I am intercepting the send and receive calls in using PMPI interface. My goal is to reduce the network traffic. So I decide to compress the buffer A wanted to send by intercepting MPI_Send call. After compression, instead of sending the n bytes that A originally wanted to send, I am sending rn bytes. Where r is the compression ratio and r<1.

Since B is using MPI_Probe, B is seeing that the receive call was getting rn bytes and allocates buffer for rn bytes. But what I want to do is intercept the MPI_Recv and decompress rn bytes in n bytes and fill the buffer where B wanted those data.

Now my question is, how can I tell B that although I am sending rn bytes, I want it to allocate memory for n bytes? I don't have access to A or B's codes. All I can do is using the PMPI interface.

Shafi
  • 33
  • 4

1 Answers1

3

The message size information comes out of MPI_Probe and MPI_Recv as part of the MPI_Status object. MPI_Status contains three publicly accessible fields, namely MPI_SOURCE, MPI_TAG, and MPI_ERROR. Beyond that it contains additional information, among it the actual message size, but the implementation is opaque and one should not access directly the private fields as those are very implementation- and version-specific. The MPI standard provides a set of "getters" and "setters" that should be used to extract or to set the number of message elements. The getters are well known, e.g. MPI_Get_count, MPI_Get_elements, etc. There is only one setter:

MPI_STATUS_SET_ELEMENTS(status, datatype, count)

You should be able to use MPI_BYTE to set the correct message size in bytes.

Note that you should be able to recover the size of the uncompressed message even before actually receiving it, i.e. you should either somehow include it in the message envelope (strongly implementation-dependent) or transmit it via a separate hidden message (introducing additional latency and thus possibly negating the gains from using compression).

In any case, there are several projects trying to accomplish transparent compression of MPI data using PMPI. You should take a look at e.g. that one and possibly try to connect with the people behind it. They might be willing to share their code or at least some of the implementation details.

Hristo Iliev
  • 72,659
  • 12
  • 135
  • 186
  • Thanks for the response. I tried to contact them earlier but never heard back from them. One possible route I was thinking was to intercept MPI_Get_Count/elements and then caching the receive temporarily. So if a subsequent receive is being called, I can just copy the temp buffer into the allocated buffer. But that also rises the question of case when MPI_Recv is called without get_count or get_elements. – Shafi Oct 14 '15 at 06:37
  • Oh, _mea culpa_. I have completely overlooked most of the (obscure) information in chapter "External Interfaces". It is actually possible to portably set the count value in the status. I'm editing the answer accordingly. – Hristo Iliev Oct 14 '15 at 07:50