Cant send the whole vector through MPI_send

Question

I have been trying to learn MPI. and when i try to run the following code i get the wrong output.

if (world_rank == 0){

    vector<vector<double> > n(4,vector<double>(4));

    srand(time(NULL));

    for(int i=0; i<4 ;i++){
        for(int j=0;j<4;j++){
            n[i][j] = (double)rand()/RAND_MAX;
            cout << n[i][j] << " ";
        }
        cout << endl;
    }
    MPI_Send((void*)&n[0][0],16*sizeof(double),MPI_BYTE,1,0,MPI_COMM_WORLD);
}else{
    MPI_Status status;

    vector<vector<double> > n(4,vector<double>(4));

    MPI_Probe(0,0,MPI_COMM_WORLD,&status);

    int size;

    MPI_Get_count(&status,MPI_BYTE,&size);

    cout << endl << size << endl;

    MPI_Recv((void*)&n[0][0],16*sizeof(n[0][0]),MPI_BYTE,0,0,MPI_COMM_WORLD,MPI_STATUS_IGNORE);

    cout.flush();
    cout << endl;

    for(int i=0; i<4 ;i++){
        for(int j=0;j<4;j++){
            cout << n[i][j] << " ";
        }
        cout << endl;
    }
}

i get all the double values except the last 3. like this.

0.824468 0.752417 0.757125 0.470763 
0.251683 0.703306 0.157991 0.764423 
0.815327 0.0402807 0.897109 0.313816 
0.997203 0.796665 0.0522305 0.797733 

128

0.824468 0.752417 0.757125 0.470763 
0.251683 0.703306 0.157991 0.764423 
0.815327 0.0402807 0.897109 0.313816 
0.997203 0 0 0

can anyone tell me why this is happening? i ran the same code around a hundred times and still get the same output (of course with different values) but the last three always are 0.

but when i changed the size from 16 to 19 i get all the values.

i also have another doubt. some times the outputs (values from node 0 and 1) get overlapped. can anyone tell me how to stop that or at least explain why that happens. i mean even though send and recv are blocking functions. how can the node 1's output get printed before node 0's

score 3 · Accepted Answer · edited May 23 '17 at 12:07

3

Your definition of the 2D data n as vector<vector<double> > makes it non-contiguous in memory. Therefore, you cannot transmit it simply using MPI (there are ways for doing it, but you'd better just making the memory contiguous instead).

To have your memory contiguous, you can declare your n like this (not tested):

vector<double> ndata(4*4); //contiguous storage of the actual data
vector<double*> n(4);      //vector of pointers to access the actual data
for (int i=1; i<4; i++)    //initialisation of the pointers to the data
    n[i] = &ndata[4*i];

Of course, there are better ways of defining a contiguous storage for multidimensional arrays in C++, but this is just a quick fix for your immediate problem. See for example this answer for a better structure.

And BTW, your MPI_Send() and MPI_Recv() calls should use 4*4 MPI_DOUBLE instead of 4*4*sizeof(double) MPI_BYTE.

edited May 23 '17 at 12:07

Community

1
1

answered Mar 02 '16 at 08:28

Gilles

9,269
4
34
53

thanks, i Got it, i was just experimenting with it. Can you tell me about the other thing i asked ( which is why the output of both nodes overlap on the screen.). Is there a way to stop it. i get that both nodes might be at different parts of the code at different times, but since both the send and recv commands are blocking. how can something being written after receiving print earlier than something that is printed before sending. If you can answer this i can rest in peace. thanks. – Adithya Sama Mar 02 '16 at 08:51
For the interleaved printings, it's just a matter of race conditions between the various processes, and buffering of the output by the OS. For a bit more details, see [this](http://stackoverflow.com/a/33665044/5239503) – Gilles Mar 02 '16 at 09:03
i read your link. i am aware of that part. but like said, i need some kind of mutual exclusion or barrier to control right?. that's what's happening here. i print before sending data and then again print after receiving it. so only after node 0 prints will i be sending data (during this node 1 stays blocked), and after node 1 receive's data it can print the sent data. but sometimes i get the sent data printed before the node 0's print. – Adithya Sama Mar 02 '16 at 10:49
You cannot force the ordering of printing in parallel. Even with some barriers to synchronise you processes, the OS / fs / whatever can reorder it... So unless you print into a file and use MPI-IO to make sure the various contributions from different threads are placed in the right locations, you cannot enforce ordering of printing... – Gilles Mar 02 '16 at 11:04

Cant send the whole vector through MPI_send

1 Answers1