4

I'm trying to implement HAL in Go, just to see if I can. This means that I've got a HAL type that is generic over the payload, and also contains the _links:

type HAL[T any] struct {
    Payload T
    Links   Linkset `json:"_links,omitempty"`
}

In the HAL spec, the payload is actually at the top level and not nested inside it - like, e.g. Siren would be. So that means given the following:

type TestPayload struct {
    Name   string `json:"name"`
    Answer int    `json:"answer"`
}

    hal := HAL[TestPayload]{
        Payload: TestPayload{
            Name:   "Graham",
            Answer: 42,
        },
        Links: Linkset{
            "self": {
                {Href: "/"},
            },
        },
    }

The resulting JSON should be:

{
    "name": "Graham",
    "answer": 42,
    "_links": {
      "self": {"href": "/"}
    }
}

But I can't work out a good way to get this JSON marshalling to work.

I've seen suggestions of embedding the payload as an anonymous member, which works great if it's not generic. Unfortunately, you can't embed generic types in that way so that's a non-starter.

I probably could write a MarshalJSON method that will do the job, but I'm wondering if there's any standard way to achieve this instead?

I've got a Playground link with this working code to see if it helps: https://go.dev/play/p/lorK5Wv-Tri

Cheers

blackgreen
  • 34,072
  • 23
  • 111
  • 129
Graham
  • 4,095
  • 4
  • 29
  • 37

4 Answers4

0

Yes, unfortunately you can't embed the type parameter T. I'll also argue that in the general case you shouldn't attempt to flatten the output JSON. By constraining T with any, you are admitting literally any type, however not all types have fields to promote into your HAL struct.

This is semantically inconsistent.

If you attempt to embed a type with no fields, the output JSON will be different. Using the solution with reflect.StructOf as an example, nothing stops me from instantiating HAL[[]int]{ Payload: []int{1,2,3}, Links: ... }, in which case the output would be:

{"X":[1,2,3],"Links":{"self":{"href":"/"}}}

This makes your JSON serialization change with the types used to instantiate T, which is not easy to spot for someone who reads your code. The code is less predictable, and you are effectively working against the generalization that type parameters provide.

Using the named field Payload T is just better, as:

  • the output JSON is always (for most intents and purposes) consistent with the actual struct
  • unmarshalling also keeps a predictable behavior
  • scalability of the code is not an issue, as you don't have to repeat all of the fields of HAL to build an anonymous struct

OTOH, if your requirements are precisely to marshal structs as flattened and everything else with a key (as it might be the case with HAL types), at the very least make it obvious by checking reflect.ValueOf(hal.Payload).Kind() == reflect.Struct in the MarshalJSON implementation, and provide a default case for whatever else T could be. Will have to be repeated in JSONUnmarshal.

Here is a solution with reflection that works when T is not a struct and scales when you add more fields to the main struct:

// necessary to marshal HAL without causing infinite loop
// can't declare inside the method due to a current limitation with Go generics
type tmp[T any] HAL[T]

func (h HAL[T]) MarshalJSON() ([]byte, error) {
    // examine Payload, if it isn't a struct, i.e. no embeddable fields, marshal normally
    v := reflect.ValueOf(h.Payload)
    if v.Kind() == reflect.Pointer || v.Kind() == reflect.Interface {
        v = v.Elem()
    }
    if v.Kind() != reflect.Struct {
        return json.Marshal(tmp[T](h))
    }

    // flatten all fields into a map
    m := make(map[string]any)
    // flatten Payload first
    for i := 0; i < v.NumField(); i++ {
        key := jsonkey(v.Type().Field(i))
        m[key] = v.Field(i).Interface()
    }
    // flatten the other fields
    w := reflect.ValueOf(h)
    // start at 1 to skip the Payload field
    for i := 1; i < w.NumField(); i++ {
        key := jsonkey(w.Type().Field(i))
        m[key] = w.Field(i).Interface()
    }
    return json.Marshal(m)
}

func jsonkey(field reflect.StructField) string {
    // trickery to get the json tag without omitempty and whatnot
    tag := field.Tag.Get("json")
    tag, _, _ = strings.Cut(tag, ",")
    if tag == "" {
        tag = field.Name
    }
    return tag
}

With HAL[TestPayload] or HAL[*TestPayload] it outputs:

{"answer":42,"name":"Graham","_links":{"self":{"href":"/"}}}

With HAL[[]int] it outputs:

{"Payload":[1,2,3],"_links":{"self":{"href":"/"}}}

Playground: https://go.dev/play/p/bWGXWj_rC5F

blackgreen
  • 34,072
  • 23
  • 111
  • 129
0

I'd make a custom JSON codec that inserts _links field at the end of the JSON jenerated for the payload.

Marshaller.


type Link struct {
    Href string `json:"href"`
}

type Linkset map[string]Link

type HAL[T any] struct {
    Payload T
    Links   Linkset `json:"_links,omitempty"`
}

func (h HAL[T]) MarshalJSON() ([]byte, error) {
    payloadJson, err := json.Marshal(h.Payload)
    if err != nil {
        return nil, err
    }
    if len(payloadJson) == 0 {
        return nil, fmt.Errorf("Empty payload")
    }
    if h.Links != nil {
        return appendField(payloadJson, "_links", h.Links)
    }
    return payloadJson, nil
}

func appendField[T any](raw []byte, fieldName string, v T) ([]byte, error) {
    // The JSON data must be braced in {}
    if raw[0] != '{' || raw[len(raw)-1] != '}' {
        return nil, fmt.Errorf("Not an object: %s", string(raw))
    }
    valJson, err := json.Marshal(v)
    if err != nil {
        return nil, err
    }
    // Add the field at the end of the json text
    result := bytes.NewBuffer(raw[:len(raw)-1])
    // Append `"<fieldName>":value`
    // Insert comma if the `raw` object is not empty
    if len(raw) > 2 {
        result.WriteByte(',')
    }
    // tag
    result.WriteByte('"')
    result.WriteString(fieldName)
    result.WriteByte('"')
    // colon
    result.WriteByte(':')
    // value
    result.Write(valJson)
    // closing brace
    result.WriteByte('}')
    return result.Bytes(), nil
}

The marshaller returns an error if Payload serializes to something other than JSON object. The reason is that the codec can add _links field to objects only.

Unmarshaller:

func (h *HAL[T]) UnmarshalJSON(raw []byte) error {
    // Unmarshal fields of the payload first.
    // Unmarshal the whole JSON into the payload, it is safe:
    // decorer ignores unknow fields and skips "_links".
    if err := json.Unmarshal(raw, &h.Payload); err != nil {
        return err
    }
    // Get "_links": scan trough JSON until "_links" field
    links := make(Linkset)
    exists, err := extractField(raw, "_links", &links)
    if err != nil {
        return err
    }
    if exists {
        h.Links = links
    }
    return nil
}

func extractField[T any](raw []byte, fieldName string, v *T) (bool, error) {
    // Scan through JSON until field is found
    decoder := json.NewDecoder(bytes.NewReader(raw))
    t := must(decoder.Token())
    // should be `{`
    if t != json.Delim('{') {
        return false, fmt.Errorf("Not an object: %s", string(raw))
    }
    t = must(decoder.Token())
    if t == json.Delim('}') {
        // Empty object
        return false, nil
    }
    for decoder.More() {
        name, ok := t.(string)
        if !ok {
            return false, fmt.Errorf("must never happen: expected string, got `%v`", t)
        }
        if name != fieldName {
            skipValue(decoder)
        } else {
            if err := decoder.Decode(v); err != nil {
                return false, err
            }
            return true, nil
        }
        if decoder.More() {
            t = must(decoder.Token())
        }
    }
    return false, nil
}

func skipValue(d *json.Decoder) {
    braceCnt := 0
    for d.More() {
        t := must(d.Token())
        if t == json.Delim('{') || t == json.Delim('[') {
            braceCnt++
        }
        if t == json.Delim('}') || t == json.Delim(']') {
            braceCnt--
        }
        if braceCnt == 0 {
            return
        }
    }
}

The unmarshaller fails on non-object as well. It is required to read _links field. For that the input must be an object.

The full example: https://go.dev/play/p/E3NN2T7Fbnm

func main() {
    hal := HAL[TestPayload]{
        Payload: TestPayload{
            Name:   "Graham",
            Answer: 42,
        },
        Links: Linkset{
            "self": Link{Href: "/"},
        },
    }
    bz := must(json.Marshal(hal))
    println(string(bz))

    var halOut HAL[TestPayload]
    err := json.Unmarshal(bz, &halOut)
    if err != nil {
        println("Decode failed: ", err.Error())
    }
    fmt.Printf("%#v\n", halOut)
}

Output:

{"name":"Graham","answer":42,"_links":{"self":{"href":"/"}}}
main.HAL[main.TestPayload]{Payload:main.TestPayload{Name:"Graham", Answer:42}, Links:main.Linkset{"self":main.Link{Href:"/"}}}
Pak Uula
  • 2,750
  • 1
  • 8
  • 13
  • this is an acceptable solution in theory, if the complexity of the marshal/unmarshal is warranted for the use, however it's hard to get right. In fact your code panics if `HAL` is instantiated with something other than a struct – blackgreen Oct 16 '22 at 07:45
  • @blackgreen sure thing it fails. There is no way to add `_links` field to anything other than an object. And there is no way to extract `_links` from non-objects. What do you mean by *complexity*? This codec is much simpler compared to the `json.Decoder` – Pak Uula Oct 16 '22 at 08:48
-1

Yes, embedding is the easiest way, and as you wrote, you can't currently embed a type parameter.

You may however construct a type that embeds the type param using reflection. We may instantiate this type and marshal it instead.

For example:

func (hal HAL[T]) MarshalJSON() ([]byte, error) {
    t := reflect.StructOf([]reflect.StructField{
        {
            Name:      "X",
            Anonymous: true,
            Type:      reflect.TypeOf(hal.Payload),
        },
        {
            Name: "Links",
            Type: reflect.TypeOf(hal.Links),
        },
    })

    v := reflect.New(t).Elem()
    v.Field(0).Set(reflect.ValueOf(hal.Payload))
    v.Field(1).Set(reflect.ValueOf(hal.Links))

    return json.Marshal(v.Interface())
}

This will output (try it on the Go Playground):

{"name":"Graham","answer":42,"Links":{"self":{"href":"/"}}}

See related: Adding Arbitrary fields to json output of an unknown struct

icza
  • 389,944
  • 63
  • 907
  • 827
-1

Keep it simple.

Yes it would be nice to embed the type - but since it's not currently possible (as of go1.19) to embed a generic type - just write it out inline:

body, _ = json.Marshal(
    struct {
        TestPayload
        Links       Linkset `json:"_links,omitempty"`
    }{
        TestPayload: hal.Payload,
        Links:       hal.Links,
    },
)

https://go.dev/play/p/8yrB-MzUVK-

{
    "name": "Graham",
    "answer": 42,
    "_links": {
        "self": {
            "href": "/"
        }
    }
}

Yes, the constraint type needs to be referenced twice - but all the customization is code-localized, so no need for a custom marshaler.

colm.anseo
  • 19,337
  • 4
  • 43
  • 52
  • 3
    The `,inline` option is unknown to the `encoding/json` library. What makes this work is purely the fact of embedding. – icza Oct 15 '22 at 17:05
  • Ah! Needed in `yaml` - good to know. – colm.anseo Oct 15 '22 at 17:16
  • by embedding `TestPayload` explicitly, the marshalling is not generic anymore, while `HAL[T]` in the OP is generic – blackgreen Oct 16 '22 at 08:00
  • To use generic types one must instantiate the type i.e `HAL[TestPayload]`. The above is more wordy, yes - repeating the type definition - but essentially the same: providing a concrete type at compilation-time. Given the current embedding constraints, it's the closest the OP can get. – colm.anseo Oct 16 '22 at 12:39