1

In this thread it has been explained in two ways how to pass messages using MPI with declared data types. I have a data structure with allocatables

  type t_example 
      real, allocatable :: x(:), y(:), z(:)
  end type

For maintability of the code would the easiest thing not be to use MPI_TYPE_CONTIGUOUS as follows 

  ! -- declare 
   type(t_example) :: p1 
   type(MPI_DATATYPE) :: mpi_dtexample

  (...) 

  call MPI_TYPE_CONTIGUOUS(sizeof(p1), MPI_BYTE, mpi_dtexample, ierr);
  call MPI_TYPE_COMMIT(mpi_dtexample, ierr)

Following this I can simply use the send/recv with mpi_dtexample as being the data type.

I cannot come to my mind when it becomes more sensible to use the mpi_type_create_struct, as this would require you to explicitly tell the sequence of the declared type, with the data type and their corresponding sizes.

YES, the MPI_TYPE_CONTIGUOUS approach assumes that the declared type is contiguous and I would not be able to use this approach if I wanted to pass certain strided elements of the declared type.

Is there else anything I should raise my alarm bells on when using MPI_TYPE_CONTIGUOUS

A FULL EXAMPLE

Running with 2 ranks only.

 module md_

   use mpi_f08

   integer numtasks, rank, tag, i,  ierr
   type(mpi_status) stat


   type T_PART
      real, allocatable :: x(:), y(:), z(:)
   end type

contains


end module

program struct
   use md_
   implicit none
   type(t_part)       :: test_
   type(mpi_datatype) :: mpidt
   integer            :: sz, szz

   call MPI_INIT(ierr)
   call MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierr)
   call MPI_COMM_SIZE(MPI_COMM_WORLD, numtasks, ierr)

   tag = 1

   szz= 10
   allocate(test_% x(szz),test_% y(szz), test_% z(szz) )




   sz = sizeof(test_)
   call MPI_Type_contiguous(sz, MPI_BYTE ,mpidt, ierr)
   call MPI_TYPE_COMMIT(mpidt, ierr)



   if (rank .eq. 0) then
      do i=1,szz
         test_%x(i) = i*1+mod(i,3)
         test_%y(i) = i*2+mod(i,3)
         test_%z(i) = i*3+mod(i,3)
      end do

      call MPI_SEND(test_, 1, mpidt, 1, tag, &
                    MPI_COMM_WORLD, ierr)

   else
         call MPI_RECV(test_, 1, mpidt, 0, tag, &
                    MPI_COMM_WORLD, stat, ierr)
   endif

   print *, 'rank= ',rank,' test_% x= ', test_%z(1) ! seg faults for rank 2


   call mpi_barrier(MPI_COMM_WORLD, ierr)
   ! free datatype when done using it
   call MPI_TYPE_FREE(mpidt, ierr)
   call MPI_FINALIZE(ierr)


   end
ATK
  • 1,296
  • 10
  • 26
  • If you use `MPI_Type_contiguous()`, you won't be able to run your app in an heterogeneous environment (e.g. send from a little endian node and receive on a big endian node). Also, you are more likely to manually add some padding if the Fortran compiler chooses to do so. – Gilles Gouaillardet Oct 10 '19 at 08:19
  • What is the reason for that? And how likely is a heterogeneous HPC ? – ATK Oct 10 '19 at 08:23
  • Unlikely heterogeneous, and your MPI library is unlikely to support it. You might want to write portable code though. – Gilles Gouaillardet Oct 10 '19 at 09:12
  • 1
    Long story short, you need to know a type size in order to convert it to/from an endian neutral format, and you cannot do it if you are using byte streams. – Gilles Gouaillardet Oct 10 '19 at 09:14
  • 1
    I know there's been work on this so I may not be up to date, but with the above what do you think you are sending? Given the arrays are allocatable my gut feeling is that you will not be sending the contents of the arrays, but rather whatever the implementation uses in the derived type to tell the process where to find the memory, possible a descriptor - not the original had statically sized arrays. Also contiguous will send any padding that may be in the above type as you've put it (I'm guessing at the non-standard sizeof function), mpi_type_struct need not do so. – Ian Bush Oct 10 '19 at 10:04
  • Spot on! I clearly overlooked the `allocatable` attribute. `MPI_Type_contiguous()` is no more an option, and you have to use one derived datatype per *struct*. – Gilles Gouaillardet Oct 10 '19 at 10:38
  • Will this not work if I allocate the members before sending/receiving? – ATK Oct 10 '19 at 11:13
  • Nope, you have to allocate the members, and then create the derived datatype, commit it and use it for the send/recv operation. – Gilles Gouaillardet Oct 10 '19 at 12:31
  • I just did that. I initially allocate all members on all ranks, then I do the mpi_type_contiguous and commit. Now after I sent the declared type from one to another rank, and try to print it on the receiving rank I get a seg fault?. what am I missing – ATK Oct 10 '19 at 12:49
  • I have put a working example which reproduces the seg fault – ATK Oct 10 '19 at 12:58
  • 1
    I'm 99% sure contiguous won't work for this as when you allocate the member of the derived type who knows in memory where that will be. To be honest I'd keep life simple and just send the component members separately. – Ian Bush Oct 10 '19 at 13:06
  • Thank you. It is a petty though. I will do many of these send forth and back. The problem is that every now and then, a new feature is added which means more members of the declared type is added, which means I will end up modify the send/recv procedures – ATK Oct 10 '19 at 13:44
  • Don't think you can avoid that - even with type_struct you would have to add new members. Encapsulate the comms of these beasts in a couple of routines and modify those is about the best you can do – Ian Bush Oct 10 '19 at 15:14
  • because of `allocatable`, you cannot use `MPI_Type_contiguous()` but you have to stick to `MPI_Type_create_struct()`. you still have to use one type per object, so sending members independently might not be as bad as it seems. – Gilles Gouaillardet Oct 11 '19 at 02:25
  • The interesting part which I don't understand is if I had removed the allocatable and made my declared type allocatable instead, i.e. having an array of structs, that seemed to work – ATK Oct 11 '19 at 06:59
  • Array of structs will be contiguous in memory. A struct of dynamically sized arrays will almost certainly be not. – Ian Bush Oct 11 '19 at 09:07
  • Yes that makes sense. As you know a struct with dynamically allocated sizes will have each array being Contiguous. I was thinking that each allocated array was followed by the next in memory. I.e struct% x(1:n) will be followed by struct% y(1:n) in memory. Is this not guaranteed? If not, why would the standard not enforce it? – ATK Oct 12 '19 at 10:25

0 Answers0