4

i am calling a nested for loop as follows:

do ir = 1,Nr
    do iom = iom1, iom2
       xyz(1) = xo(1) + xom(iom)*r
       xyz(2) = xo(2) + yom(iom)*r
       xyz(3) = xo(3) + zom(iom)*r
       call FUNSUB(xyz,Nprop,PropXYZ)
    enddo
enddo

where FUNSUB evaluates a property in the following manner:

id = 1
do l=0,lmax
    do m=-l,l
        id = id + 1
        Prop(id) = RHO * Ylm(l,m,xl(1),xl(2),xl(3))
    enddo
enddo

now i am trying to parallelize this with something of the form

!$OMP parallel do reduction(+:Prop) private(ir, l, m, j, iom, r, wrad, xyz, PropOm, PropOm1, PropXYZ)

where some of the other private variables seen here are left out of my example code for brevity. i am intentionally trying to make l and m private so that inside the FUNSUB loops, the properties are evaluated independently. i find, however, that they are shared between threads. i have put in some debugging options and found that id continually goes above the bounds of Prop and have found that l also goes about lmax as well as m being outside the bounds of (-l,l).

my question is: how do i go about making sure that l and m are kept private and NOT shared from the main routine? it seems that the problem is that it is an entirely different subroutine which holds these values and that, somehow these private declarations of variables do not carry over

drjrm3
  • 4,474
  • 10
  • 53
  • 91

2 Answers2

3

I think your problem is that in Fortran, local subroutine variables can be static and that the !$OMP does not propagate to called functions.

Your code should work as expected if you copy-paste FUNSUB into the calling loops, i.e. if you manually inline it.

Update

After reading-up a bit on the looniness of how local variables may or may not be implemented by the compiler, I guess your best option is to declare FUNSUB as RECURSIVE. This forces all local variables onto the stack, meaning that each OpenMP thread which calls FUNSUB will have its own, private, set of local variables.

Community
  • 1
  • 1
Pedro
  • 1,344
  • 9
  • 17
  • i knew it was because i was calling an external function, but i want to find a way around inlining it because FUNSUB is actually a function handle passed to the subroutine and may be a different subroutine on different calls. – drjrm3 Jun 23 '12 at 16:34
  • @Laurbert515: If your function is a function handle, then you're kind of stuck as OpenMP statements don't propagate to subroutines. How should the compiler know about that when compiling your subroutine? The only way around this is to either declare the variables in `FUNSUB` with the *opposite* of `SAVE` attribute, although I don't even know if this exists, or to pass them on as parameters to that function, which will be massively inefficient. There may be a compiler option that does the former. – Pedro Jun 23 '12 at 16:41
  • @Laurbert515: Scratch the previous comment, the `RECURSIVE` keyword should solve all your problems. I have updated my answer accordingly. – Pedro Jun 23 '12 at 16:47
1

The full code of the FUNSUB would have to be investigated. Correct usage of an external function in an OpenMP region should not be a problem. There should be no reason and it is not possible to declare variables in scope of an external function as private.

Modern Fortran compilers will not create local variables of a function static, if you don't tell it to do so. This may be done using a compiler switch (big NO NO in a parallel code) or by declaring the variables as SAVE as has been save. Note that all initialized variables are SAVE implicitly. That means that:

   integer :: local = 0

is SAVE even without the SAVE keyword and will be shared among threads.

It is a good practice to have all functions called in parallel declared as PURE. I am even for having all non-recursive functions at least effectively PURE.

Vladimir F Героям слава
  • 57,977
  • 4
  • 76
  • 119