Append to golang slice passed as empty interface

Question

How to append to empty interface (that has been verified to be a *[]struct)?

func main() {
  var mySlice []myStruct // myStruct can be any struct (dynamic)
  decode(&mySlice, "...")
}

func decode(dest interface{}, src string) {
  // assume dest has been verified to be *[]struct
  var modelType reflect.Type = getStructType(dest)
  rows, fields := getRows(src)
  for _, row := range rows {
    // create new struct of type modelType and assign all fields
    model := reflect.New(modelType)
    for field := fields {
      fieldValue := getRowValue(row, field)
      model.Elem().FieldByName(field).Set(fieldValue)
    }
    castedModelRow := model.Elem().Interface()
      
    // append model to dest; how to do this?
    // dest = append(dest, castedModelRow)
  }
}

Things I've tried:

This simply panics: reflect: call of reflect.Append on ptr Value (as we pass &mySlice instead of mySlice)

dest = reflect.Append(reflect.ValueOf(dest), reflect.ValueOf(castedModelRow))

This works but doesn't set the value back to dest... in main func, len(mySlice) remains 0 after decode function is called.

func decode(dest interface{}, src string) {
  ...
  result := reflect.MakeSlice(reflect.SliceOf(modelType), rowCount, rowCount)
  for _, row : range rows {
    ...
    result = reflect.Append(result, reflect.ValueOf(castedModelRow))
  }
  dest = reflect.ValueOf(result)
}

score 2 · Accepted Answer · answered Nov 07 '21 at 02:49

Here's how to fix the second decode function shown in the question. The statement

dest = reflect.ValueOf(result)

modifies local variable dest, not the caller's value. Use the following statement to modify the caller's slice:

reflect.ValueOf(dest).Elem().Set(result)

The code in the question appends decoded elements after the elements created in reflect.MakeSlice. The resulting slice has len(rows) zero values followed by len(rows) decoded values. Fix by changing

result = reflect.Append(result, reflect.ValueOf(castedModelRow))

to:

result.Index(i).Set(model)

Here's the update version of the second decode function in the question:

func decode(dest interface{}, src string) {
    var modelType reflect.Type = getStructType(dest)
    rows, fields := getRows(src)
    result := reflect.MakeSlice(reflect.SliceOf(modelType), len(rows), len(rows))
    for i, row := range rows {
        model := reflect.New(modelType).Elem()
        for _, field := range fields {
            fieldValue := getRowValue(row, field)
            model.FieldByName(field).Set(fieldValue)
        }
        result.Index(i).Set(model)
    }
    reflect.ValueOf(dest).Elem().Set(result)
}

Run it on the Playground.

Thanks a ton, this works like a charm! Really appreciate your explanation and for you to kindly point out the append issue. This well written answer on reflection certainly suit your username handle :) — kent-id, Nov 07 '21 at 03:19

score 1 · Answer 2 · answered Nov 07 '21 at 03:14

You were very close with your original solution. You had to de-reference the pointer before calling the append operation. This solution would be helpful if your dest already had some existing elements and you don't want to lose them by creating a newSlice.

tempDest := reflect.ValueOf(dest).Elem()
tempDest = reflect.Append(tempDest, reflect.ValueOf(model.Interface()))

Similar to how @I Love Reflection pointed out, you finally need to set the new slice back to the pointer.

reflect.ValueOf(dest).Elem().Set(tempDest)

Overall Decode:

var modelType reflect.Type = getStructType(dest)
rows, fields := getRows(src)
tempDest := reflect.ValueOf(dest).Elem()
for _, row := range rows {
    model := reflect.New(modelType).Elem()
    for _, field := range fields {
        fieldValue := getRowValue(row, field)
        model.FieldByName(field).Set(fieldValue)
    }
    tempDest = reflect.Append(tempDest, reflect.ValueOf(model.Interface()))
}
reflect.ValueOf(dest).Elem().Set(tempDest)

In this case I don't want to append to existing values in dest but rather override it, and yep: reflect.ValueOf(dest).Elem().Set(result) does the trick. Thanks a lot! I initially tried reflect.ValueOf(dest).Set(result) before asking this question... and got panic: reflect.Value.Set using unaddressable value. Tho this seems to be because func (v Value) Set(x Value) assigns v.ptr = x.ptr so that's why we need to do .Elem() so the Set function receives the value instead of ptr? — kent-id, Nov 07 '21 at 03:29

Append to golang slice passed as empty interface

2 Answers2