2

I've encountered some strange behavior with reflect.DeepEqual. I have an object of type map[string][]string, with one key whose value is an empty slice. When I use gob to encode this object, and then decode it into another map, these two maps are not equal according to reflect.DeepEqual (even though the content is identical).

package main

import (
    "fmt"
    "bytes"
    "encoding/gob"
    "reflect"
)

func main() {
    m0 := make(map[string][]string)
    m0["apple"] = []string{}

    // Encode m0 to bytes
    var network bytes.Buffer
    enc := gob.NewEncoder(&network)
    enc.Encode(m0)

    // Decode bytes into a new map m2
    dec := gob.NewDecoder(&network)
    m2 := make(map[string][]string)
    dec.Decode(&m2)

    fmt.Printf("%t\n", reflect.DeepEqual(m0, m2)) // false
    fmt.Printf("m0: %+v != m2: %+v\n", m0, m2) // they look equal to me!
}

Output:

false
m0: map[apple:[]] != m2: map[apple:[]]

A couple notes from follow-up experiments:

If I make the value of m0["apple"] a nonempty slice, for example m0["apple"] = []string{"pear"}, then DeepEqual returns true.

If I keep the value as an empty slice but I construct the identical map from scratch rather than with gob, then DeepEqual returns true:

m1 := make(map[string][]string)
m1["apple"] = []string{}
fmt.Printf("%t\n", reflect.DeepEqual(m0, m1)) // true!

So it's not strictly an issue with how DeepEqual handles empty slices; it's some strange interaction between that and gob's serialization.

icza
  • 389,944
  • 63
  • 907
  • 827
rampatowl
  • 1,722
  • 1
  • 17
  • 38
  • Guessing it's something to do with a `nil` slice and an empty slice being treated the same in most places, but possibly not within `DeepEqual`. – Adrian Jun 06 '18 at 15:41

1 Answers1

2

This is because you encode an empty slice, and during decoding the encoding/gob package only allocates a slice if the one provided (the target to decode into) is not big enough to accomodate the encoded values. This is documented at: gob: Types and Values:

In general, if allocation is required, the decoder will allocate memory. If not, it will update the destination variables with values read from the stream.

Since there are 0 elements encoded, and a nil slice is perfectly capable of accomodating 0 elements, no slice will be allocated. We can verify this if we print the result of comparing the slices to nil:

fmt.Println(m0["apple"] == nil, m2["apple"] == nil)

Output of the above is (try it on the Go Playground):

true false

Note that the fmt package prints nil slice values and empty slices the same way: as [], you cannot rely on its output to judge if a slices is nil or not.

And reflect.DeepEqual() treats a nil slice and an empty but non-nil slice different (non-deep equal):

Note that a non-nil empty slice and a nil slice (for example, []byte{} and []byte(nil)) are not deeply equal.

icza
  • 389,944
  • 63
  • 907
  • 827
  • Thanks for the explanation! So the main problem is that gob treats nil slices and empty slices the same, but DeepEqual does not. Do you think there is a good reason for this, or should one of the packages be changed? Wondering if I should submit a git issue – rampatowl Jun 06 '18 at 15:56
  • @rampatowl `gob` does not treat `nil` and empty slices the same, but it does not allocate a slice if the "existing" one is enough to hold the encoded elements. If this is an issue for you, use `nil` slices in place of empty ones before encoding. I don't think these are flawed. `gob` does so to minimize allocations (and therefore improve performance). – icza Jun 06 '18 at 15:57
  • Gotcha, thanks. I'm surprised that the encoding works when a map value is `nil`, because in the case where the value type is a pointer, this gives me a `gob: encodeReflectValue: nil element` error. In case you have any insights about this contradiction, I posted a separate question here: https://stackoverflow.com/questions/50725102/gob-cant-encode-map-with-a-nil-pointer-value – rampatowl Jun 06 '18 at 16:19