0

I'm using go-hdf5 and I'm hitting a problem when trying to write attributes in a loop from a map.

The attributes are created correctly (correct name and datatype) but the written value is garbage.

The same code outside of the loop works fine. I tried both the v := v idiom and wrapping the code in a closure to capture v but it doesn't make a difference.

Here is the gist of the code (error checking intentionally left out for clarity):

m := map[string]interface{"foo", 42}
for k, v := range m {
    // [...]
    v := v
    attr.Write(&v, dtype)
}

The Write method is using reflection to grab a pointer to the value and forwards it to the C library. The relevant part of the code is just:

func (s *Attribute) Write(data interface{}, dtype *Datatype) error {
    v := reflect.ValueOf(data)
    addr := unsafe.Pointer(v.Pointer())
    return h5err(C.H5Awrite(s.id, dtype.id, addr))
}

If I replace the map by a slice of interface{}, I get the exact same problem so my hunch is that this has to do with the binding of loop variables, but yet v := v doesn't help so I'm not sure.

I'm quite familiar with Go, HDF5 (C library) and go-hdf5 but I'm really stuck here. Any idea?

BTW I'm using go1.5.1 darwin/amd64.

Simon
  • 31,675
  • 9
  • 80
  • 92
  • What is `dtype`? I suspect your problem is due to `H5Awrite` interpreting `v` as something other than it is. In particular it's probably looking for an integer or something like that, and getting an interface value instead. – hobbs Feb 26 '16 at 18:09
  • `dtype` is an `hdf5.Datatype` of the correct type for the value of the interface. It seems that type casting `v` to the correct type (replacing `v := v` by `v := v.(int)` for instance) works. Of course I'm not supposed to know the correct type (other than through reflection). – Simon Feb 26 '16 at 18:14
  • First, are you certain that you're getting the pointer to the correct structure? You're calling `Write` with a pointer to an interface, which is almost always incorrect. Second, is `Write` storing a reference to the pointer? If you don't maintain a reference to the pointer in Go, after you pass it to C, the value of `v` is going to be GC'ed. (also, run with go1.6 which adds additional safety checks to the cgo code) – JimB Feb 26 '16 at 18:16
  • The C library either flushes or keeps its own copy of the value so I'm pretty sure GC is not the problem. I think you are right, I need to provide a pointer to the value itself. I probably need to use reflection then. – Simon Feb 26 '16 at 18:18
  • oh yes, I didn't read the second part of the example closely enough. You're definitely just using a pointer to the interface, and not the value. – JimB Feb 26 '16 at 18:20
  • @JimB is correct, you need to provide a pointer to your value. You're getting garbage because it's writing out a pointer to your interface. If you could provide what your Datatype is, it'd be a great help in figuring out how HDF5 is going to interpret your buffer. I would use a map with a struct that has the correct layout, instead of a map to an interface which is a reference type made up of two -- 32 or 64 bit -- words (type, value). – Denzel Feb 26 '16 at 18:31
  • A *nice* way to handle this might be to patch go-hdf5 `Write` to recognize when `v.Kind()` is `reflect.Interface` and if so use `e.Elem().Pointer()`. Then an interface could be passed without `&` and the right thing would happen provided that `dtype` matched the thing contained within the interface. – hobbs Feb 26 '16 at 18:57
  • Failing that you should be able to do about the same thing externally. – hobbs Feb 26 '16 at 18:58
  • Thank you all for your comments, I posted an answer using reflection. I may make a PR for go-hdf5 if I have time. – Simon Feb 26 '16 at 19:49

1 Answers1

0

The Write method needs a pointer to a value, not a pointer to an interface containing the value. You can get it using reflection:

u := reflect.New(reflect.ValueOf(v).Type())
u.Elem().Set(reflect.ValueOf(v))
v := u.Interface()
Simon
  • 31,675
  • 9
  • 80
  • 92