2

I'm doing a little MPI (openmpi) program in C for a workshop at college. Our objective is to observe the time difference between the two main protocols of MPI, eager and rendezvous, regarding the message size.

We haven't worked with MPI before, and we have thought that there may be a way to "select" between the two protocols. Searching on google for information about how to do it I found (somewhere I don't remember) that there is an eager limit. I read that it is set by the MPI implementation, and also, that you can change it somehow.

Any advice on how to choose between the protocols? Are there any relation between the protocols and MPI_Send/MPI_Isend?

I thought that changing the receiver buffer size will break from eager and start using rendezvous. But it's just a hunch.

Here is my code for now:

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include "mpi.h"


#define KBdata 32000 //openmpi default buffer size
#define ndata KBdata/4 //number of ints that fits in buffer

int main(int argc, char *argv[]) {

    int myid, numprocs;
    int tag,source,destination,count;
    int buffer[ndata];
    MPI_Status status;
    MPI_Request request;

    int iter = 20;

    MPI_Init(&argc,&argv);
    MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
    MPI_Comm_rank(MPI_COMM_WORLD,&myid);

    if (myid == 0 && numprocs == 2) { //to 
        int recvID = 1;
        double acum = 0;
        int i;
        double startT;
        for (i=0;i<iter;++i)
        {
            double startTime = MPI_Wtime();

            MPI_Send(&buffer,ndata,MPI_INT,recvID,0,MPI_COMM_WORLD);

            double endTime = MPI_Wtime();
            double elapsed = endTime - startTime;
            acum += elapsed;
            printf("%d, %f, elapsed: %f\n",i,acum,elapsed);fflush(stdout);
            usleep(500000);
        }
        printf("total: %f\nmean: %f\n", acum, acum/iter);   
    }
    else if (numprocs == 2) {
        int i;
        for(i=0; i<iter; ++i)
        {
            printf("Waiting for receive\n");fflush(stdout);
            MPI_Recv(&buffer,ndata,MPI_INT,0,0,MPI_COMM_WORLD,&status);
            printf("Received %d\n",i);fflush(stdout); 
        }
    }
    else {
        printf("Need only 2 threads\n");
    }

    MPI_Finalize();


    return 0;
}

Thank you in advice.

Hristo Iliev
  • 72,659
  • 12
  • 135
  • 186
ShakMR
  • 115
  • 2
  • 10
  • Buffer sizes and protocols are specific for each type of network interconnect supported by Open MPI and selectable by passing the corresponding MCA parameters. Use `ompi_info --param btl ` where `` is the interconnect type, e.g. `openib`, `tcp`, etc. to get a list of all parameters for the given BTL. – Hristo Iliev Dec 16 '14 at 21:17
  • In MPICH you would set this with an environment variable. Look for 'EAGER' in the output of mpivars. Cray MPI, though based on MPICH, uses a slightly different environment variable mechanism. – Rob Latham Dec 17 '14 at 15:15

1 Answers1

0

There is no direct connection between eager/rendezvous and MPI_Send/Isend. However, if you're under the eager limit, your MPI_Send is no longer blocking. If you want it to block regardless, you can use MPI_Ssend.

Regarding eager limits:

MVAPICH2:
  MV2_IBA_EAGER_THRESHOLD= < nbytes >
Intel MPI (depending on version):
  I_MPI_EAGER_THRESHOLD= < nbytes >
  I_MPI_SHM_EAGER_THRESHOLD= < nbytes >
Open MPI:
  --mca_btl_openib_eager_limit < nbytes >
  --mca_btl_openib_rndv_eager_limit < nbytes >
Cray MPICH:
  MPICH_GNI_MAX_EAGER_MSG_SIZE=<value>
Victor Eijkhout
  • 5,088
  • 2
  • 22
  • 23