MPI - Sending segments of an array

Question

So I have an array of doubles. I would like to send, say every 5th double, to the receiving process. So essentially, I need a way of sending specific doubles with strides between them. Is there a function to do this, apart from storing the doubles into a send buffer? Would it be better to make my own derived type?

score 7 · Accepted Answer · answered Mar 18 '13 at 18:24

You should definitely create an MPI data type; it gives the MPI library an opportunity to avoid the extra copy for marshaling from the array, and it's pretty straightforward to do using, in this case, MPI_Type_vector():

#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>

int main(int argc, char** argv)
{
    int size, rank;
    const int bigsize=50;
    const int stride = 5;
    const int count = (bigsize + stride - 1)/stride;

    const int sender = 0;
    const int receiver = 1;
    const int mytag = 1;

    MPI_Init(&argc,&argv);
    MPI_Comm_size(MPI_COMM_WORLD,&size);
    MPI_Comm_rank(MPI_COMM_WORLD,&rank);

    if (size < 2) {
        fprintf(stderr,"%s: Require at least two processors.\n", argv[0]);
        MPI_Finalize();
        exit(-1);
    }


    if(rank == sender)
    {
        double bigarray[bigsize];
        for (int i=0; i<bigsize; i++)
            bigarray[i] = 0.;

        for (int i=0; i<bigsize; i+=stride)
            bigarray[i] = i/stride;

        printf("[%d]: ", rank);
        for (int i=0; i<bigsize; i++)
            printf("%lf ", bigarray[i]);
        printf("\n");

        MPI_Datatype everyfifth;

        MPI_Type_vector( count, 1, stride, MPI_DOUBLE, &everyfifth);
        MPI_Type_commit(&everyfifth);

        MPI_Send(bigarray, 1, everyfifth, receiver, mytag, MPI_COMM_WORLD);

        MPI_Type_free(&everyfifth);
    }
    else if( rank == receiver )
    {
        double littlearray[count];

        MPI_Status status;

        MPI_Recv(littlearray, count, MPI_DOUBLE, sender, mytag,
                    MPI_COMM_WORLD, &status);

        printf("[%d]: ", rank);
        for (int i=0; i<count; i++)
            printf("%lf ", littlearray[i]);
        printf("\n");
    }

    MPI_Finalize();

    return 0;
}

Compiling and running gives

$ mpicc -o vector vector.c -std=c99
$ mpirun -np 2 ./vector
[0]: 0.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 0.000000 2.000000 0.000000 0.000000 0.000000 0.000000 3.000000 0.000000 0.000000 0.000000 0.000000 4.000000 0.000000 0.000000 0.000000 0.000000 5.000000 0.000000 0.000000 0.000000 0.000000 6.000000 0.000000 0.000000 0.000000 0.000000 7.000000 0.000000 0.000000 0.000000 0.000000 8.000000 0.000000 0.000000 0.000000 0.000000 9.000000 0.000000 0.000000 0.000000 0.000000 
[1]: 0.000000 1.000000 2.000000 3.000000 4.000000 5.000000 6.000000 7.000000 8.000000 9.000000

One question - I did this, however it is slower than sending the whole array. Would you care to explain why? Also is there a more efficient way, since I am using this in a loop with many thousands of iterations. — JessMcintosh, Mar 18 '13 at 21:03
It will depend. If the data is small either way - so the length of the message doesn't contribute significantly to the communications time - then the additional overhead to selecting out just the relevant bits may be noticable; that will depend on the network layer. There's also the overhead of actually creating, commiting and freeing the type, but that only has to be done once, not once per message, so you can amortize that over many iterations. — Jonathan Dursi, Mar 18 '13 at 21:08

MPI - Sending segments of an array

1 Answers1

Linked