Julia: Accessing DArray in a function on a processor

Question

I create a DArray:

d = dzeros(3)

Now I would like to run a function in parallel using pmap. I would like that function to access whatever part of d is local on the current processor. Something like

function foo()
    global d
    a = localpart(d)
    a[1] = 1
end

However, I get

exception on 2: exception on 4: ERROR: d not defined
 in mcmc_sub! at /home/benjamin/.julia/v0.3/Mamba/src/model/mcmc.jl:67
 in anonymous at multi.jl:847
 in run_work_thunk at multi.jl:613
 in anonymous at task.jl:847

on each process.

d should be defined on each processor. For example code like this works:

julia> d = dzeros(3)
3-element DArray{Float64,1,Array{Float64,1}}:
 0.0
 0.0
 0.0

julia> @spawnat(2, (a = localpart(d); a[1]=1;))
RemoteRef(2,1,65)

julia> d
3-element DArray{Float64,1,Array{Float64,1}}:
 1.0
 0.0
 0.0

I think you're more likely to get an answer if you open an issue. — StefanKarpinski, Jul 13 '14 at 19:59

score 1 · Accepted Answer · answered Jul 14 '14 at 03:45

I'm not totally sure no copy happens, but my understanding is that you can just pass d as a parameter (it is a reference to the whole thing, passing it won't move the data).

Simple example:

function foo(d, u)
    r, = myindexes(d)
    return u * 100000 + sum(d[r])
end

function main()
    d = distribute(1:100)
    show(pmap(x-> foo(d, x), 1:10))
end

# julia -p 2 -L test.jl -e "main()"

I not sure whether you can assign to the distributed array in this way, you probably want to create a new one (piece-by-piece); this is what is done in the cellular automaton example.

Julia: Accessing DArray in a function on a processor

1 Answers1