Send function pointer via MPI

Question

Is it safe to pass function pointers via MPI as a way of telling another node to call a function? Someone may say that Passing any kind of pointers via MPI is meaningless, but I wrote some code to verify it.

//test.cpp
#include <cstdio>
#include <iostream>
#include <mpi.h>
#include <cstring>

using namespace std;

int f1(int a){return a + 1;}
int f2(int a){return a + 2;}
int f3(int a){return a + 3;}

using F=int (*)(int);

int main(int argc, char *argv[]){
    MPI_Init(&argc, &argv);
    int rank, size;
    MPI_Status state;
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    //test
    char data[10];
    if( 0 == rank ){
        *(reinterpret_cast<F*>(data))=&f2;
        for(int i = 1 ; i < size ; ++i)
            MPI_Send(data, 8, MPI_CHAR, i, 0, MPI_COMM_WORLD);
    }else{
        MPI_Recv(data, 8, MPI_CHAR, 0, 0, MPI_COMM_WORLD, &state);
        F* fp = reinterpret_cast<F*>(data);
        int ans = (**fp)(10);
        cout << ans << endl;
    }


    MPI_Finalize();
    return 0;
}

Here is the output:

I ran it via MVAPICH, and it works well. But I just don't now why since separate address spaces means that the pointer value is USELESS in any process other than the one that generated it.

P.S. here is my hostfile

blade11:1
blade12:1
blade13:1
blade14:1
blade15:1
blade16:1
blade17:1
blade18:2
blade19:1

and I ran mpiexec -n 10 -f hostfile ./test, and compiled it using C++11

Hristo Iliev · Accepted Answer · 2016-10-11T11:22:25.453

You are lucky in the sense that your cluster environment is homogeneous and no address space randomisation for ordinary executables is in place. As a consequence, all images are loaded at the same base address and laid out similarly in memory, hence functions have the same virtual addresses in all MPI ranks (note that this is rarely true for symbols from dynamically linked libraries as those are usually loaded at random addresses).

If you compile the source twice using different compilers or using the same compiler but with different compiler options, then have some ranks run the first executable and the rest run the second one, the program will definitely crash.

Try this:

$ mpicxx -std=c++11 -O0 -o test_O0 test.cpp
$ mpicxx -std=c++11 -O2 -o test_O2 test.cpp
$ mpiexec -f hostfile -n 5 ./test_O0 : -n 5 ./test_O2
12
12
12
12
<crash>

The different levels of optimisation result in function code of different size in test_O0 and test_O2. Consequently, f2 will no longer have the same virtual address in all ranks. The ranks that run the same executable as rank 0 will print 12, while the rest will segfault.

score 2 · Answer 2 · answered Oct 11 '16 at 11:20

Is it safe to pass function pointers via MPI as a way of telling another node to call a function?

No, it is not. Address space is not shared among processes.

However, MPI processes which are the result of programs built from the same source can be organised to call a specific function when a certain message is received:

char data = 0;
MPI_Recv(data, 1, MPI_CHAR, 0, 0, MPI_COMM_WORLD, &state);

if (data == 255) { 
    f2(10); /* and so forth */
}

score 0 · Answer 3 · answered Jan 20 '17 at 08:34

No.

However there is trick involving macros that map a certain codification of a function to a local function pointer/callback that can be recognized in all processes uniformly. For example, this is used in HPX http://stellar.cct.lsu.edu/files/hpx_0.9.5/html/HPX_PLAIN_ACTION.html to run a function across inhomogeneous systems.

Send function pointer via MPI

3 Answers3