Trying to wrap my head around some nonblocking mpi behaviour

Question

So I've been trying some things in MPI to get a feel for it and I've produced some results that I can't explain by combing through the documentation. I'm using open mpi.

#include <mpi.h>
#include <stdio.h>
#include <unistd.h>

#define PROBLEMSIZE 41 
int main(int argc, char** argv) {
    // Initialize the MPI environment
    MPI_Init(&argc, &argv);

    // Get the number of processes
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    // Get the rank of the process
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

    double send[PROBLEMSIZE];
    MPI_Request request[PROBLEMSIZE];
    int flag;
    int array_of_indices[PROBLEMSIZE];
    MPI_Status status[PROBLEMSIZE];

    if(world_rank == 0)
    {
        for (int i = 0; i < PROBLEMSIZE; i++)
        {
            request[i] = MPI_REQUEST_NULL;
            send[i] = i;
        }

        for (int i = 0; i < PROBLEMSIZE; i++)
        {
            MPI_Isend(&send[i],1,MPI_DOUBLE,1,i,MPI_COMM_WORLD,&request[i]);
        }
        MPI_Waitall(PROBLEMSIZE,request,status);
        for (int i = 0; i < PROBLEMSIZE; i++)
        {
            //printf("%i count %i", i, status[i].count);
            //printf("%i cancel %i", i, status[i].cancel);
            printf("%i MPI_SOURCE %i\n", i, status[i].MPI_SOURCE);
            printf("%i MPI_TAG %i\n", i, status[i].MPI_TAG);
            printf("%i MPI_ERROR %i\n \n", i, status[i].MPI_ERROR);
        }
        printf("thread 0 done");
    }

    if (world_rank == 1)
    {  
        sleep(10);
        for (int i = 0; i < PROBLEMSIZE; i++)
        {
            MPI_Irecv(&send[i],1,MPI_DOUBLE,0,i,MPI_COMM_WORLD, &request[i]);
        }
        int outcount;
        
 
        do {
            MPI_Testsome(PROBLEMSIZE,request, &outcount, array_of_indices, MPI_STATUSES_IGNORE);
            printf("did some waiting, outcount is %i\n",outcount);
        } while (outcount != 0);



        for (int i = 0; i < PROBLEMSIZE; i++)
        {
            printf("%f \n",send[i]);
        }
        
        /*
        sleep(20);

        MPI_Testall(PROBLEMSIZE,request,MPI_STATUSES_IGNORE);

        for (int i = 0; i < PROBLEMSIZE; i++)
        {
            printf("%f \n",send[i]);
        }
        */
    }



    MPI_Finalize();
    return 0;
}

/Edit

What I'm trying to achieve is being able to send an arbitrary amount of data with different tags from one process to the other, where the sender is fine to block and wait until it's received, but the receiver shouldn't block when there's nothing to receive.

Edit/

So here's the things that trip me up:

1. MPI_Waitall on line 36 doesn't actually seem to block until the communication happens. Consequently the printing of the status afterwards yields complete bogus. (Source = -2, Tag = -1, Error = 0). The next line however printf("thread 0 done"); does seem to wait until the program is done, which is more than a bit confusing to me.

2. When trying to receive the numbers on line 59, only the first 16 seem to make the trip, I tried using MPI_Testall, MPI_Testsome and MPI_Waitsome for that all with the same result. I assumed this was due to the buffer hitting some kind of limit, and tried to loop it multiple times as long as it was receiving things. According to the documentation outcount should contain the number of received things, but is always 0 when trying it with MPI_Testsome, and -32766 when trying it with MPI_Waitsome. Both of these are obviously suspicious numbers, but I'm not sure how or why it would manage to overflow to -32766.

3. Using MPI_Waitall to receive the data in thread 1 yields the desired behaviour, casting doubt on my suspicion that it's an issue with the buffer. However that implementation isn't what I'm looking for, since I would want thread 1 to be able to do stuff, and periodically check for new data coming in.

Answer 1

1. I am not sure the MPI_TAG and MPI_SOURCE fields are relevant when the MPI_Status is related to a send request
2. MPI_Test() and friends are non blocking subroutines that return immediately. I am not sure what you are trying to achieve here, but you might want to use MPI_Waitsome() instead.