Domain resizing on an array of records hangs

Question

I have a hypergraph data structure with two arrays, one for edges, and one for vertices (similar to bipartite graph). I have a problem with resizing the arrays, so I tried the simplified example:

ar dom = {0..0};

var arr: [dom] int;

writeln(arr);

dom = {0..#2};

writeln(arr);

dom = {0..#1};

writeln(arr);

record Vertex {}
record Edge   {}

record Wrapper {
  type nodeType;
  type idType;
  var id: idType;
}

record NodeData {
  type nodeIdType;

  var ndom = {0..-1};
  var neighborList: [ndom] nodeIdType;

  proc numNeighbors() return ndom.numIndices;
  var lock$: sync bool = true;

  // This method is not parallel-safe                                                                                                                                                                                                                                           
  proc addNodes(vals) {
    lock$; // acquire lock                                                                                                                                                                                                                                                      
    neighborList.push_back(vals);
    lock$ = true; // release the lock                                                                                                                                                                                                                                           
  }

  proc readWriteThis(f) {
    f <~> new ioLiteral("{ ndom = ") <~> ndom <~> new ioLiteral(", neighborlist = ") <~> neighborList <~> new ioLiteral(", lock$ = ") <~> lock$.readFF() <~> new ioLiteral(" }");
  }
}

type vType = Wrapper(Vertex, int);
type eType = Wrapper(Edge, int);

var dom1 = {0..0};
var arr1: [dom1] NodeData(eType);

writeln(arr1);

dom1 = {0..#2};

writeln(arr1);

dom1 = {0..#1};

writeln(arr1);

When I try to run this code, it hangs with the following output:

$ ./resize -nl 1
salloc: Granted job allocation 15015
0
0 0
0
{ ndom = {0..-1}, neighborlist = , lock$ = true }

So the resizing on an array of integers works perfectly fine, but I can't resize my array of records. What am I doing wrong?

As a side note, when I try to do resize domains in my complete code, I see the domains changing, but my arrays that use the domains do not change at all. At least the code does not hang though.

EDIT

I tried another example that actually illuminates my original problem better:

class Test {
  var dom;

  var ints: [dom] int;

  proc resize(size) {
    dom = {dom.low..size};
  }
}

var test = new Test(dom = {0..-1});
writeln(test);
test.resize(1);
writeln(test);

Here is the output that I see:

$ ./resize -nl 1
salloc: Granted job allocation 15038
{dom = {0..-1}, ints = }
{dom = {0..1}, ints = }
salloc: Relinquishing job allocation 15038

So my problem is that the resize method is useless. It does change the domain, but it does not change the member array.

Answer 1

The following is in response to the problem you're seeing in your EDIT example:

I'm afraid that you're getting caught in a dark corner of the compiler as of version 1.17, and I regret that it exists, though I think we can get you out of it.

Starting with some background and important context: Since its outset, Chapel has supported constructors on classes and records (e.g., proc C(...) for a class C), yet these were naive in their design, particularly w.r.t. generic classes and records. Over the past several releases, we've been shifting from constructors to initializers (e.g., proc init(..) for a class C) to address these limitations.

As of today's release, version 1.17, initializers are in fairly good shape (e.g., I now use them for all new code I write and seldom swear at them), but if you provide neither an initializer nor a constructor (as in your examples), the compiler will create a default constructor (rather than a default initializer) and thus can run afoul of some of these long-standing problems. For version 1.18, the goal is to have the compiler create initializers by default and to deprecate constructors completely.

So, here are some ways to work around the issue for your smaller test program in the EDIT, all of which seem to produce the right output for me in version 1.17 of Chapel:

1) Make the class less generic. Here, I've given the dom field an initial value such that the compiler can determine its type, and this apparently helps it with the default constructor enough that it generates the expected output:

class Test {
  var dom = {0..-1};

  var ints: [dom] int;

  proc resize(size) {
    dom = {dom.low..size};
  }
}

var test = new Test(dom = {0..-1});
writeln(test);
test.resize(1);
writeln(test);

2) Write an explicit initializer. Here, I'm leaving dom generic but creating an initializer that assigns it to match the signature of your new call:

class Test {
  var dom;

  var ints: [dom] int;

  proc init(dom: domain(1)) {
    this.dom = dom;
  }

  proc resize(size) {
    dom = {dom.low..size};
  }
}

var test = new Test(dom = {0..-1});
writeln(test);
test.resize(1);
writeln(test);

3) (last resort) Request that the compiler create a default initializer (rather than a default constructor) for you. This approach really isn't intended for end-users, won't work for all cases, and will go away in the future, but may be handy to know about in the meantime. Here, I'm attaching a pragma to the class to tell the compiler to create a default initializer rather than a default constructor. Though the compiler won't create default initializers by default, for many classes and records it is able to if you ask it to, and this happens to be one of them:

pragma "use default init"
class Test {
  var dom;

  var ints: [dom] int;

  proc resize(size) {
    dom = {dom.low..size};
  }
}

var test = new Test(dom = {0..-1});
writeln(test);
test.resize(1);
writeln(test);

In the interest of space, I've only addressed your shorter example here, not your longer one, but hope that these techniques will help with it as well (and am happy to spend more time on the longer one if needed).

Answer 2

The following is in response to the problem you reported seeing in your original example (different from the issue in your EDIT example, answered separately):

The reason your original code is failing is due to the default assignment that Chapel creates for records. Specifically, given a record R:

record R {
  var x: t;
  var y: t2;
  var z: t3;
}

if you have not created an assignment between Rs, the compiler will create one of the general form:

proc =(ref lhs: R, rhs: R) {
  lhs.x = rhs.x;
  lhs.y = rhs.y;
  lhs.z = rhs.z;
}

In your example, this is causing problems because currently (as of Chapel 1.17.0) when an array is reallocated, its elements are first default constructed/initialized and then assigned. Thus, the synchronized lock$ field is getting default initialized to true and then when array elements are being copied, the assignment operator tries to re-assign that field, but it's already full.

One solution, which seems to have permitted your code to compile and run for me is to implement your own assignment function such that it doesn't touch the lock$ field:

proc =(ref lhs: NodeData(?t), rhs: NodeData(t)) {
  lhs.ndom = rhs.ndom;
  lhs.neighborList = rhs.neighborList;
  //  let's not touch lhs.lock$                                                         
}

But you should obviously think about whether this would be correct locking discipline for your use case. If you think something needs to change here w.r.t. initialization/assignment semantics w.r.t. array resizing, please feel free to file a Chapel GitHub issue against it.