2

... and it's driving me nuts trying to understand what I'm doing wrong!

Playground: https://go.dev/play/p/ZQP8Y-gwihQ

The example looks contrived but it's drawn from code that I have where the error arose. In my code I'm hashing the bytes buffer and want the process to be predictable.

package main

import (
    "bytes"
    "encoding/gob"
    "fmt"
    "log"
)

type Foo struct {
    Bar string
    Baz string
}

func (f *Foo) X() string {
    var b bytes.Buffer
    s := struct {
        Bar string
        Baz string
    }{
        f.Bar,
        f.Baz,
    }
    log.Printf("%v", s)
    gob.NewEncoder(&b).Encode(s)
    return fmt.Sprintf("%x", b)
}

func (f *Foo) Y(x string) string {
    var b bytes.Buffer
    s := struct {
        Bar string
        Baz string
        S   string
    }{
        f.Bar,
        f.Baz,
        x,
    }
    log.Printf("%v", s)
    gob.NewEncoder(&b).Encode(s)
    return fmt.Sprintf("%x", b)
}

func main() {
    a := &Foo{
        Bar: "bar",
        Baz: "baz",
    }

    log.Println(a.X())
    log.Println(a.Y("something"))
}

Running yields:

{bar baz}
{1cff81030102ff820001020103426172010c00010342617a010c0000000dff820103626172010362617a00 0 0}
{bar baz something}
{22ff83030102ff840001030103426172010c00010342617a010c00010153010c00000018ff840103626172010362617a0109736f6d657468696e6700 0 0}

Commenting out log.Println(a.X()) yields:

{bar baz something}
{22ff81030102ff820001030103426172010c00010342617a010c00010153010c00000018ff820103626172010362617a0109736f6d657468696e6700 0 0}

I expect the two encodings to the same but they differ (predictably) in locations that I assume correspond to field boundaries:

22
ff83 # 81
030102

ff84 # 82
0001030103426172010c00010342617a010c00010153010c00000018

ff84 # 82
0103626172010362617a0109736f6d657468696e6700

Even though the details differ the behavior is consistent with my code.

I'm creating a new bytes.Buffer and gob.NewEncoder in each method and so it's unclear why invoking X changes the result of Y.

DazWilkin
  • 32,823
  • 5
  • 47
  • 88
  • What is your question? – Hymns For Disco Dec 25 '21 at 00:32
  • Why does the encoding change if I comment out the `X` method. I should be encoding the same struct with a new encoder but, depending on whether I run `X`, the result changes. – DazWilkin Dec 25 '21 at 00:36
  • Is there any particular reason why you think they ought to be the same? Or are you trying to investigate how gob is implemented for curiosity's sake? – Hymns For Disco Dec 25 '21 at 00:52
  • Invoking method `X` does not change any subsequent behavior. The two encoded values are clearly different (2 fields vs 3). Running your playground example with or without the method call to `X` yields the same result for method `Y` regardless. – colm.anseo Dec 25 '21 at 01:47
  • Not for me, it doesn't. If I run without `X`, the array begins `22-ff-81`. If I run with `X`, the array begins `22-ff-83`. There are 3 byte differences. – DazWilkin Dec 25 '21 at 02:07
  • @HymnsForDisco I've been (!?) using gob to serialize a struct that I then hash. I want to use the hash as a deterministic identifier for instances of the struct. – DazWilkin Dec 25 '21 at 02:11

1 Answers1

3

What you're missing is that the stream of bytes produced by an Encoder instance has global (program-wide) state in addition to the per-Encoder state. That global state consists of [note: edited phrase here] registered-and-sent types.

When you send a typed value, if the type has not yet been registered before being sent, it will be registered for you, in the global state. This assigns an internal numeric value to the type. See Register (and its companion RegisterName). When you call your X, that registers the anonymous struct type that holds s in X. When you call your Y, that registers the anonymous struct type that holds s in Y. These get different internal type numbers. By not calling X, that type is never registered, and Y's type gets registered under the first available number.

In my code I'm hashing the bytes buffer ...

That's not a great idea for what are now probably obvious reasons. However, if you explicitly register each type in a known order, you'll be safe enough here unless some future version changes the wire format for some (presumably good) reason. Oops, testing this shows it doesn't help either. That's because even if the type is registered, it doesn't have a transmit number set until the first time a value of that type gets encoded. So you need to encode (and optionally then just discard) a value of each type.

Here is a functioning example of carefully discard-encoding the two types, so that commenting out the call to log.Println(a.X()) has no effect on the encoding of the second value.

torek
  • 448,244
  • 59
  • 642
  • 775