make openmp loop to combine small arrays into one big array

Question

I have this loop which looks like the following in openmp but I am getting allocation errors:

!$OMP PARALLEL DO PRIVATE(fn)
  do fn=0, NFILES 
    allocate(temparray(1:3,1:Ntot))
    LOAD temparray FROM FILE(fn)
    Ntot = len(temparray)
    TOTALARRAY(1+nstart:nstart+Ntot,1:3)=temparray(1:Ntot, 1:3)
    deallocate(temparray)
    nstart=nstart+len(temparray)
  end do
!$OMP END PARALLEL DO

Is there a way to load these files into memory and combine them very quickly using openmp? Or do I have to do it sequentially? The arrays I have are MUCH larger than this - this is just for demonstration. I want to load them in in parallel.

I don't understand your code. What is `LOAD temparray FROM FILE(fn)`? You can only call `len` with character arguments. Are `A`, `B` and `C` loaded from files? They are 1-dimensional arrays but a 2-d array is allocated. The `TOTALARRAY` assignment depends on `nstart`, which is redefined inside the loop. You would have to do an `ORDERED` openMP loop if you want to depend on the order of elements in the result. — sigma, Dec 21 '12 at 22:25
Sigma, I removed the abc as it was confusing. temparray is loaded from file. they are 3xNtot arrays. I just want to combine arrays in pieces in 4 different files into one big array as quickly as possible. I suppose it is impossible because each of the files are of different length and need to know where in TOTALARRAY to populate. — Griff, Dec 23 '12 at 03:34

score 2 · Answer 1 · answered Dec 21 '12 at 23:05

Ntot and temparray should be private. Otherwise you are allocating it and deallocating it and rewriting it from different threads and that must end up in an error. Instead of len you probably wanted size, but it will just return 3*Ntotin this case.

As @sigma pointed out, the place in totalarray where you insert the just read one depends on the progress of reading. That will not work out. You have to make it somehow predictable. Maybe just read the file and get the positions in one pass and then rewind and make a second pass with the real read. Or just use constant arrays, if it is possible. (One could also use pack at the end, if necessary, but it could take long.)

make openmp loop to combine small arrays into one big array

1 Answers1