2

I have a parallel fortran application and I would like to read in medium-to-large data arrays that will be "mirrored" on all processors. In other words, this is "global" data such as observation locations which is necessary for all processing elements to have to do their work, and not the distributed arrays that will be distributed across processors.

Is there a pure HDF solution that will efficiently read in all of the data and broadcast it to the processing elements?

I understand from the HDF parallel documentation that I can select a hyperslab in parallel using the following:

 ! open the file, etc. 
 ! ...
 call h5sselect_hyperslab_f (slabspace, H5S_SELECT_SET_F, &
     &                      data_offset, data_count, hdf_error)
 call h5pcreate_f(H5P_DATASET_XFER_F, plist_id, hdf_error)
 call h5pset_dxpl_mpio_f(plist_id, H5FD_MPIO_COLLECTIVE_F, &
     &                        hdf_error)
 call h5dread_f(dset_tb, H5T_IEEE_F32LE, obs, dims,  &
     &   hdf_error, file_space_id = slabspace, mem_space_id = memspace,  &
     &   xfer_prp = plist_id)

I understand this could be used to set the data_offset and data_count through a hyperslab, which could distribute the data to different processors.

However, I was wondering if I had missed something for my case where all of the processors will get all of the data. If I make the data_offset equal to 0 for h5sselect_hyperslab_f and ran with all of the data, am I correct to expect the performance to be terrible?

Or do I need to break up the I/O on the processors myself with a hyperslab then use an MPI allgather?

Menos
  • 43
  • 5
  • 1
    I would say that the simplest thing to do would be to read your data with only one process and then to broadcast it to the others; that way, you would avoid contentions you could have on your file system with all processes accessing the same file at the same time. – MBR Mar 26 '19 at 15:47
  • 1
    Thanks MBR. If the file system isn't parallel, that's likely to be the best solution. For a parallel file system I wonder if I can do better with the hyperslabs. – Menos Mar 27 '19 at 02:22
  • 1
    Your performance will likely be limited to the striping of your file in a parallel file system. You could indeed ensure to have a striping larger than 1, and then to read `n` hyperslabs (with `n` the number of stripes or a multiple of the number of stripes) and gather/broadcast them. If the number of processes you have is much larger than the number of the striping of your file (that is likely) and you want all of them to read a slab, you will still have contention issues. It would be worth estimating if it is worth the pain to implement such an I/O strategy. – MBR Mar 28 '19 at 15:27

0 Answers0