As with your previous question, the answer to this question depends on the specific domain map. The Chapel language and compiler expect a domain map (and its implementation of domains and arrays) to support a standard interface, but how it implements that interface is completely up to its author. This interface includes things like "allocate a new domain for me", "allocate a new array over that domain for me", "iterate over the indices/elements of the domain/array", "randomly access the array", etc. Thus, a given domain map implementation may be very space efficient and minimal, or it can allocate everything on every locale redundantly, as its author thinks best.
That said, if we consider standard domain maps like Block
, they behave the way you would expect: E.g., for a {1..n, 1..n} array mapped across 4 locales, each locale will store ~(n**2 / 4)
elements of the array rather than all n**2
elements. A random access to that array will be implemented by determining which locale owns the element and having the compiler/runtime manage the communication required to get at that remote element (as implemented by the domain map). Information is stored redundantly when it only requires O(1) storage, since this redundancy is better than communicating to get the values. E.g., each locale would store the {1..n, 1..n} bounds of the domain/array since it is cheaper to store those bounds than to communicate with some centralized location to get them.
This is one of those cases where a picture can be worth a thousand words. Taking a look at the slides for the talks where we introduced these concepts (like slide 34 of this presentation) could be much more instructive than the following text-based description.
Walking through your declarations and cleaning them up a bit, here's roughly what happens as this code is executed:
const region = {1..10,5..10},
regionbox = {1..5,1..5},
grid2d = /*a 2D arrangement of locales*/;
Nothing about these declarations refer to other locales (no on-clauses, no dmapped clauses), so these would all result in domains and arrays that are stored locally on the locale where the task encountering the declarations is executing (locale #0 at the program's start-up time).
const Space : domain(2) dmapped Block( boundingBox = regionbox,
target_locales = grid2d
) = region.
The dmapped Block(...)
clause causes an instance of the Block
domain map class to be allocated on each locale in grid2d
. Each instance of the class stores the bounding box (regionbox
) and the set of target locales. Each locale also gets an instance of a class representing the local view of the distribution named LocBlock
which stores the subset of the 2D plane which is owned by that locale as defined by the bounding box and the target locale set.
The declaration and initialization of Space
invokes a method on the current locale's copy of the Block
domain map object created in the previous step, asking it to create a new domain. This causes each locale in grid2d
to allocate a pair of classes corresponding to the global and local views of the domain, respectively. The global descriptor describes the domain's indices as a whole (e.g., region
) while the local descriptor describes that locale's personal subset of region
.
var myarray: [Space] int;
This declaration asks the current locale's copy of the global Space
domain class created in the previous step to create a new array. This causes each locale in grid2d
to allocate a pair of classes representing the global and local views of the array. The global view of the array tends not to store much state and is used primarily to dispatch methods on the array to the appropriate local descriptor. The local descriptor stores the array elements corresponding to the locale's subarray.
I hope this helps clarify the issues you are asking about.