1

I have a huge matrix and need to perform matrix operations on huge submatrices of that matrix. The matrix is stored CMOed binary in a file. There is no problem reading this matrix as a whole blockcyclic using MPI_FILE_CREATE_DARRY(), but I couldn't figure out how to do so on submatrices, so that only a certain part of the matrix lies blockcyclic in memory... Thanks for advise!

zp3
  • 5
  • 2
  • 1
    what is a 'CMOed binary'? – Rob Latham Apr 20 '15 at 18:09
  • 2
    The arguments to DARRAY are long and sometimes hard to reason about. Got some code handy for what works and what doesn't? Have you tried something and it did not work? – Rob Latham Apr 20 '15 at 18:10
  • @rob(1): CMOed: (C)olum (M)ajor (O)rder(ed), default fortran dimension nesting order – zp3 Apr 20 '15 at 18:34
  • @rob(2): I think the solution isn't anywhere in the darray() procedure; the thing is, one can just specify an offset to the begin of the file (in the mpi_read_at/write_at resp. mpi_set_file_vew functions) but one can not specify anywhere global- or subarray sizes (necessary for adjustment of the leading dimension). One could use the mpi_type_create_subarray() function but then data isn't loaded blockcyclic. The solution might come with mpi_type_create_struct() but then I fear things are becoming nasty... – zp3 Apr 20 '15 at 18:51
  • This seems like a really interesting and reasonable question, but for the life of me I can't figure out how to compose Subarray and Darray, or even how to nicely transpose them in memory. Maybe read it in as a subarray, transpose it once as blocked but non-cyclic, then again as cyclic? And yes, rolling your own is certainly possible, but that Darray type map in the standard makes my blood run cold. – Jonathan Dursi Apr 23 '15 at 00:59
  • Composing subarray and darray can't be the solution because that would need a huge amount of communication and buffer storage. – zp3 Apr 23 '15 at 14:49
  • I figured out, that it might be possible using a three times recursive mpi_struct type, but I don't know if mpi even supports recursive type creation, so for God's sake, I won't try that - dare not speaking of efficiency... – zp3 Apr 23 '15 at 14:55

0 Answers0