Why does this sample code (f90, MPI, derived types) causes invalid read/write (valgrind or dmalloc)?

Question

This is the incriminated code (it is related to another question I asked, here):

program foo

  use mpi

  implicit none

  type double_st
     sequence
     real(kind(0.d0)) :: x,y,z
     integer :: acc
  end type double_st

  integer, parameter :: n=8

  INTEGER :: MPI_CADNA_DST

  integer :: nproc, iprank
  INTEGER :: IERR, STAT(MPI_STATUS_SIZE)
  INTEGER :: MPI_CADNA_DST_TMP
  INTEGER ::&
       COUNT=4,&
       BLOCKLENGTHS(4)=(/1,1,1,1/),&
       TYPES(4)=(/MPI_DOUBLE_PRECISION,&
       MPI_DOUBLE_PRECISION,&
       MPI_DOUBLE_PRECISION,&
       MPI_INTEGER/)
  INTEGER(KIND=MPI_ADDRESS_KIND) :: DISPLS(4), LB, EXTENT
  TYPE(DOUBLE_ST) :: DST
  INTEGER :: I

  type(double_st), allocatable :: bufs(:), bufr(:)

  allocate(bufs(n), bufr(n))

  CALL MPI_INIT(IERR)
  CALL MPI_COMM_SIZE(MPI_COMM_WORLD, NPROC, IERR)
  CALL MPI_COMM_RANK(MPI_COMM_WORLD, IPRANK, IERR)

  CALL MPI_GET_ADDRESS(DST%X,   DISPLS(1))
  CALL MPI_GET_ADDRESS(DST%Y,   DISPLS(2))
  CALL MPI_GET_ADDRESS(DST%Z,   DISPLS(3))
  CALL MPI_GET_ADDRESS(DST%ACC, DISPLS(4))
  DO I=4,1,-1
     DISPLS(I)=DISPLS(I)-DISPLS(1)
  ENDDO
  CALL MPI_TYPE_CREATE_STRUCT(4,BLOCKLENGTHS,DISPLS,TYPES, MPI_CADNA_DST_TMP,IERR)
  CALL MPI_TYPE_COMMIT(MPI_CADNA_DST_TMP,IERR)

  CALL MPI_TYPE_GET_EXTENT(MPI_CADNA_DST_TMP, LB, EXTENT, IERR)
  CALL MPI_TYPE_CREATE_RESIZED(MPI_CADNA_DST_TMP, LB, EXTENT, MPI_CADNA_DST, IERR)
  CALL MPI_TYPE_COMMIT(MPI_CADNA_DST,IERR)

  bufs(:)%x=iprank
  bufs(:)%y=iprank
  bufs(:)%z=iprank
  bufs(:)%acc=iprank
  call mpi_send(bufs(1), n, mpi_cadna_dst, 1-iprank, 0, mpi_comm_world, ierr)
  call mpi_recv(bufr(1), n, mpi_cadna_dst, 1-iprank, 0, mpi_comm_world, stat, ierr)


  deallocate(bufs, bufr)

end program foo

Compiled with intelMPI, version 4.0 or 5.0, I get invalid read/write errors with valgrind or with dmalloc at the send. With openMPI, it is not that clear with that minimum example, but I got similar problems with this communication in the big code from which it is extracted.

Thanks for helping!

score 1 · Accepted Answer · answered Sep 02 '15 at 10:20

1

It looks like the use of sequence is the culprit here. Since your data are not aligned the same way, forcing the linear packing with the sequence keyword generate some unaligned accesses, probably while writing the one of the arrays. Removing it does the trick.

answered Sep 02 '15 at 10:20

Gilles

9,269
4
34
53

Ok so the problem comes from `sequence`. But then my problem is that the derived type `double_st` I use comes from a library that is not developped by me, so I cannot remove `sequence` (which must be there for some reason...). I there a way to send/receive derived sequence type with MPI – janou195 Sep 03 '15 at 07:53
I guess that you can just compute the size of the type using a `sizeof` or even a `MPI_Get_address()` on `buf(2)` minus the one of `bufs(1)` and then use a `MPI_Type_contiguous(size, MPI_BYTE, mytype, ierr)`. This way you won't have the issue of alignment internally. That said, I'm not entirely sure if the alignment issue comes from the MPI transfers or from the read/write of the actual data like in `bufs(:)%x=iprank` since there, `bufs(2)%x` is misaligned. – Gilles Sep 03 '15 at 10:58

score 0 · Answer 2 · answered Sep 02 '15 at 11:46

0

I think that he used derived-type definition with sequence(the guy who wrote the code).SEQUENCE cause the components of the derived type to be stored in the same sequence they are listed in the type definition. If SEQUENCE is specified, all derived types specified in component definitions must be sequence types.You should tell us more about compilation,are you on Linux or Windows also.

answered Sep 02 '15 at 11:46

Richard Rublev

7,718
16
77
121

I am on linux, and I compile with mpiifort. I don't get what you mean by `If SEQUENCE is specified, all derived types specified in component definitions must be sequence types` ? – janou195 Sep 03 '15 at 07:47
You can find here more: http://h21007.www2.hp.com/portal/download/files/unprot/fortran/docs/lrm/lrm0038.htm – Richard Rublev Sep 03 '15 at 07:52
But in this case, all the components are intrinsic non-derived types, no? – janou195 Sep 03 '15 at 08:01

Why does this sample code (f90, MPI, derived types) causes invalid read/write (valgrind or dmalloc)?

2 Answers2