"Merge" fields two structs of same type

Question

Looking at this struct:

type Config struct {
  path string
  id   string
  key  string
  addr string
  size uint64
}

Now I have a DefaultConfig intialized with some values and one loaded from a file, let's say FileConfig. I want both structs to me merged, so that I get a Config with the content of both structs. FileConfig should override anything set in DefaultConfig, while FileConfig may not have all fields set. (Why this? Because a potential user may not know the default value, so removing that entry would be equivalent to setting the default - I think)

I thought I'd need reflection for this:

 func merge(default *Config, file *Config) (*Config) {
  b := reflect.ValueOf(default).Elem()
  o := reflect.ValueOf(file).Elem()

  for i := 0; i < b.NumField(); i++ {
    defaultField := b.Field(i)
    fileField := o.Field(i)
    if defaultField.Interface() != reflect.Zero(fileField.Type()).Interface() {
     defaultField.Set(reflect.ValueOf(fileField.Interface()))
    }
  }

  return default
 }

Here I am not sure:

• If reflection is needed at all
• There may be easier ways to do this

Another issue I see here is that checking for zero values may be tricky: what if the overriding struct intends to override with a zero value? Luckily, I don't think it applies to my case - but this becomes a function, it may become a problem later

Answer 1

Foreword: The encoding/json package uses reflection (package reflect) to read/write values, including structs. Other libraries also using reflection (such as implementations of TOML and YAML) may operate in a similar (or even in the same way), and thus the principle presented here may apply to those libraries as well. You need to test it with the library you use.

For simplicity, the solution presented here uses the standard lib's encoding/json.

---

An elegant and "zero-effort" solution is to use the encoding/json package and unmarshal into a value of the "prepared", default configuration.

This handles everything you need:

• missing values in config file: default applies
• a value given in file overrides default config (whatever it was)
• explicit overrides to zero values in the file takes precedence (overwrites non-zero default config)

To demonstrate, we'll use this config struct:

type Config struct {
    S1 string
    S2 string
    S3 string
    S4 string
    S5 string
}

And the default configuration:

var defConfig = &Config{
    S1: "", // Zero value
    S2: "", // Zero value
    S3: "abc",
    S4: "def",
    S5: "ghi",
}

And let's say the file contains the following configuration:

const fileContent = `{"S2":"file-s2","S3":"","S5":"file-s5"}`

The file config overrides S2, S3 and the S5 fields.

Code to load the configuration:

conf := new(Config) // New config
*conf = *defConfig  // Initialize with defaults

err := json.NewDecoder(strings.NewReader(fileContent)).Decode(&conf)
if err != nil {
    panic(err)
}

fmt.Printf("%+v", conf)

And the output (try it on the Go Playground):

&{S1: S2:file-s2 S3: S4:def S5:file-s5}

Analyzing the results:

• S1 was zero in default, was missing from file, result is zero
• S2 was zero in default, was given in file, result is the file value
• S3 was given in config, was overriden to be zero in file, result is zero
• S4 was given in config, was missing in file, result is the default value
• S5 was given in config, was given in file, result is the file value

Answer 2

Reflection is going to make your code slow.

For this struct I would implement a straight Merge() method as:

type Config struct {
  path string
  id   string
  key  string
  addr string
  size uint64
}

func (c *Config) Merge(c2 Config) {
  if c.path == "" {
    c.path = c2.path
  }
  if c.id == "" {
    c.id = c2.id
  }
  if c.path == "" {
    c.path = c2.path
  }
  if c.addr == "" {
    c.addr = c2.addr
  }
  if c.size == 0 {
    c.size = c2.size
  }
}

It's almost same amount of code, fast and easy to understand.

You can cover this method with uni tests that uses reflection to make sure new fields did not get left behind.

That's the point of Go - you write more to get fast & easy to read code.

Also you may want to look into go generate that will generate the method for you from struct definition. Maybe there event something already implemented and available on GitHub? Here is an example of code that do something similar: https://github.com/matryer/moq

Also there are some packages on GitHub that I believe are doing what you want in runtime, for example: https://github.com/imdario/mergo

Answer 3

Another issue I see here is that checking for zero values may be tricky: what if the overriding struct intends to override with a zero value?

In case you cannot utilize encoding/json as pointed out by icza or other format encoders that behave similarly you could use two separate types.

type Config struct {
    Path string
    Id   string
    Key  string
    Addr string
    Size uint64
}

type ConfigParams struct {
    Path *string
    Id   *string
    Key  *string
    Addr *string
    Size *uint64
}

Now with a function like this:

func merge(conf *Config, params *ConfigParams)

You could check for non-nil fields in params and dereference the pointer to set the value in the corresponding fields in conf. This allows you to unset fields in conf with non-nil zero value fields in params.

Answer 4

This solution won't work for your specific question, but it might help someone who has a similar-but-different problem:

Instead of creating two separate structs to merge, you can have one "default" struct, and create a modifier function for it. So in your case:

type Config struct {
  path string
  id   string
  key  string
  addr string
  size uint64
}

var defcfg = Config {
  path: "/foo",
  id: "default",
  key: "key",
  addr: "1.2.3.4",
  size: 234,
}

And your modifier function:

func myCfg(c *Config) {
  c.key = "different key"
}

This works in tests where I want to test many small, different variations of a largely unmodified struct:

func TestSomething(t *testing.T) {
  modifiers := []func (*Config){
    .... // modifier functions here
  }
  for _, f := range modifiers {
    tc := defcfg // copy
    f(&tc)
    // now you can use tc.
  }
}

Not useful when you read your modified config from a file into a struct, though. On the plus side: this also works with zero values.