1

I have a Fortran 90 program that creates a large column of integers at the end of each loop. I would like to write this column into an HDF5 file with each new integer column being written as a new column in the HDF5 file. That is, at the end of the first loop, write column 1, then at the end of the second loop, write column 2, etc, until a 2D array with as many columns as there were loops is created.

Each of these integer columns will always be the same length and I know how many columns will be needed in the end. To do this, I create a dataspace large enough to hold the final 2D array. Then I select a hyperslab beginning with offset (0,0), then (0,1) for the next loop, (0,2) for the loop after that, etc., each time writing the data that my program creates.

This isn't quite working, however. Currently, with the offset (0,0) everything works. However, if I try to write to a hyperslab in any other column, I get nonsense values in my output .h5 file.

Am I implementing the hyperslab code correctly?

Here is a MWE which creates a dataspace of (20,10) and writes a column of twenty '5's into the first column of the .h5 file. If voffset is changed from (/0,0/) to (/0,1/), the '5's are not written and instead random numbers will appear in the second column of the .h5 file:

program hdf5_test
use hdf5
implicit none

! HDF5 file and path names
 character(len=10), parameter :: outfilename = "OutData.h5" ! Output file name
 character(len=8), parameter :: dsetvbinnedname = "velocity"

! Declare HDF5 IDs
integer(HID_T) :: outfile_id       ! File identifier
integer(HID_T) :: dsetv_id           
integer(HID_T) :: dataspacev_id

! Dataset dimensions in the file
integer(HSIZE_T), dimension(2) :: binned_vdims = (/20,1/)  ! Chunk dimensions
integer(HSIZE_T), dimension(2) :: export_binned_vdims = (/20,10/)  ! File dimensions

! Data buffers
integer, dimension(20) :: binned_vdata  ! Chunk dimension

! Chunk parameters
integer(HSIZE_T), dimension(2) :: voffset, vchunkcount

integer :: rank = 2
integer :: error ! HDF5 error flag

! Initialize FORTRAN interface
call h5open_f(error)

! Create a new file using the default properties.
call h5fcreate_f(outfilename, H5F_ACC_TRUNC_F, outfile_id, error)

! Create the data space for the binned dataset.
call h5screate_simple_f(rank, export_binned_vdims, dataspacev_id, error)

! Create the chunked dataset.
call h5dcreate_f(outfile_id, dsetvbinnedname, H5T_NATIVE_INTEGER, dataspacev_id, dsetv_id, error)

! Select hyperslab
voffset = (/0,0/)
vchunkcount = (/20,1/)

call h5sselect_hyperslab_f(dataspacev_id, H5S_SELECT_SET_F, voffset, vchunkcount, error)

binned_vdata = 5 ! Write a column of '5's

! Write the data to the dataset.
call h5dwrite_f(dsetv_id, H5T_NATIVE_INTEGER, binned_vdata, export_binned_vdims, error, file_space_id=dataspacev_id)

! Close dataspace, dataset, and file
call h5dclose_f(dsetv_id, error)
call h5sclose_f(dataspacev_id, error)
call h5fclose_f(outfile_id, error)

call h5close_f(error)

end program hdf5_test
EP_Guy
  • 95
  • 7

2 Answers2

1

You haven't specified a memory dataspace. With the Fortran HDF5 API, the default identifier in the absence of an explicit memory dataspace argument is H5S_ALL_F, which means that the memory dataspace will be the same as the provided file dataspace.

Consequently, when you provide an dataspace that is offset into the file, HDF5 writes values from memory that is similarly offset, and this memory is beyond the bounds of your array.

Create another dataspace for the data in memory using a similar call to h5create_simple_f as you have now, and provide that to the h5dwrite_f call as the mem_space_id argument.

IanH
  • 21,026
  • 2
  • 37
  • 59
  • 1
    This suggestion to use a memory dataspace is correct. I've supplied working code implementing this suggestion and selected it as the accepted answer. – EP_Guy Dec 04 '13 at 00:59
1

As suggested by others, I needed to use a memory dataspace. Working code, provided by Barbara at the HDF Group, is below.

program hdf5_test
use hdf5
implicit none

! HDF5 file and path names
 character(len=10), parameter :: outfilename = "OutData.h5" ! Output file name
 character(len=8), parameter :: dsetvbinnedname = "velocity"

! Declare HDF5 IDs
integer(HID_T) :: outfile_id       ! File identifier
integer(HID_T) :: dsetv_id            
integer(HID_T) :: dataspacev_id 
integer(HID_T) :: memspace 

! Dataset dimensions in the file
integer(HSIZE_T), dimension(2) :: binned_vdims = (/20,1/)  ! Chunk dimensions
integer(HSIZE_T), dimension(2) :: export_binned_vdims = (/20,10/)  ! File dimensions

! Data buffers
integer, dimension(20,1) :: binned_vdata  ! Chunk dimension

! Chunk parameters
integer(HSIZE_T), dimension(2) :: voffset, vchunkcount

integer :: rank = 2,i, j
integer :: error ! HDF5 error flag

! Initialize FORTRAN interface
call h5open_f(error)

! Create a new file using the default properties.
 call h5fcreate_f(outfilename, H5F_ACC_TRUNC_F, outfile_id, error)

! Create the data space for the binned dataset.
call h5screate_simple_f(rank, export_binned_vdims, dataspacev_id,  error)

! Create the chunked dataset.
call h5dcreate_f(outfile_id, dsetvbinnedname, H5T_NATIVE_INTEGER, dataspacev_id, dsetv_id, error)

! Create the memory space for the selection
call h5screate_simple_f(rank, binned_vdims, memspace,  error)

! Select hyperslab
voffset(1) = 0
vchunkcount = (/20,1/)


DO i = 1, binned_vdims(1) 
     DO j = 1, binned_vdims(2) 
         binned_vdata(i,j) = 5 
     END DO
END DO


DO i= 1, export_binned_vdims(2) 
  voffset(2) = i-1
  call h5sselect_hyperslab_f(dataspacev_id, H5S_SELECT_SET_F, voffset, vchunkcount, error)

  ! Write the data to the dataset.
  call h5dwrite_f(dsetv_id, H5T_NATIVE_INTEGER, binned_vdata, binned_vdims, error, memspace, dataspacev_id)
  call h5sclose_f(dataspacev_id, error)
  call h5dget_space_f(dsetv_id, dataspacev_id, error)
END DO

! Close dataspace, dataset, and file
call h5dclose_f(dsetv_id, error)
call h5sclose_f(memspace, error)
call h5fclose_f(outfile_id, error)

call h5close_f(error)

end program hdf5_test
EP_Guy
  • 95
  • 7