MPI IO Writing a file when offset is not known

Question

I want to use MPI IO to write files. The processes is in a while loop and it calls a function which generates random amount of data. I want to write this data to a single file. How can I accomplish this?

Answer 1

It might help you:

http://social.microsoft.com/Forums/en-US/47b99479-37d9-43a4-b1e3-90efd3d52c0a/can-different-mpi-processes-write-data-in-one-file-with-different-offset?forum=windowshpcmpi

But maybe you ll need to pass a variable which contains the start point of every process..

Answer 2

in each iteration of your while loop, each process knows how much data will be written. Use MPI_SCAN to share that data, then MPI_File_write_at_all to write collectively:

      incr = generate_random_data();
      MPI_Scan(&incr, &new_offset, 1, MPI_LONG_LONG_INT, 
                      MPI_SUM, MPI_COMM_WORLD);
      new_offset -= incr;

      ret = MPI_File_write_at_all(mpi_fh, new_offset, buf, count,
                              datatype, status);