In Go, how can I convert a struct to a byte array?

Question

I have an instance of a struct that I defined and I would like to convert it to an array of bytes. I tried []byte(my_struct), but that did not work. Also, I was pointed to the binary package, but I am not sure which function I should use and how I should use it. An example would be greatly appreciated.

Answer 1

One possible solution is the "encoding/gob" standard package. The gob package creates an encoder/decoder that can encode any struct into an array of bytes and then decode that array back into a struct. There's a great post, here.

As others have pointed out, it's necessary to use a package like this because structs, by their nature, have unknown sizes and cannot be converted into arrays of bytes.

I've included some code and a play.

package main

import (
    "bytes"
    "encoding/gob"
    "fmt"
    "log"
)

type P struct {
    X, Y, Z int
    Name    string
}

type Q struct {
    X, Y *int32
    Name string
}

func main() {
    // Initialize the encoder and decoder.  Normally enc and dec would be
    // bound to network connections and the encoder and decoder would
    // run in different processes.
    var network bytes.Buffer        // Stand-in for a network connection
    enc := gob.NewEncoder(&network) // Will write to network.
    dec := gob.NewDecoder(&network) // Will read from network.
    // Encode (send) the value.
    err := enc.Encode(P{3, 4, 5, "Pythagoras"})
    if err != nil {
        log.Fatal("encode error:", err)
    }

    // HERE ARE YOUR BYTES!!!!
    fmt.Println(network.Bytes())

    // Decode (receive) the value.
    var q Q
    err = dec.Decode(&q)
    if err != nil {
        log.Fatal("decode error:", err)
    }
    fmt.Printf("%q: {%d,%d}\n", q.Name, *q.X, *q.Y)
}

Answer 2

I assume you want something like the way C handles this. There is no built in way to do that. You will have to define your own serialization and deserialization to and from bytes for your struct. The binary package will help you encode the fields in your struct to bytes that you can add to the byte array but you will be responsible for specifying the lengths and offsets in the byte array that will hold the fields from your struct.

Your other options are to use one of the encoding packages: http://golang.org/pkg/encoding/ such as gob or json.

EDIT:

Since you want this for making a hash as you say in your comment the easisest thing to do is use []byte(fmt.Sprintf("%v", struct)) like so: http://play.golang.org/p/yY8mSdZ_kf

Answer 3

Just use json marshal， this is a very simple way.

newFsConfig := dao.ConfigEntity{EnterpriseId:"testing"}
newFsConfigBytes, _ := json.Marshal(newFsConfig)

Answer 4

I know this thread is old, but none of the answers were accepted, and there's a pretty simple way to do this.

https://play.golang.org/p/TedsY455EBD

important code from playground

import (
  "bytes"
  "fmt"
  "encoding/json"
)

type MyStruct struct {
  Name string `json:"name"`
}

testStruct := MyStruct{"hello world"}
reqBodyBytes := new(bytes.Buffer)
json.NewEncoder(reqBodyBytes).Encode(testStruct)

reqBodyBytes.Bytes() // this is the []byte

Answer 5

Serialization is likely proper answer.

But if you consent to unsafety and actually need to read struct as bytes, then relying on byte array memory representation might be a bit better than relying on byte slice internal structure.

type Struct struct {
    Src int32
    Dst int32
    SrcPort uint16
    DstPort uint16
}

const sz = int(unsafe.SizeOf(Struct{}))
var asByteSlice []byte = (*(*[sz]byte)(unsafe.Pointer(&struct_value)))[:]

Works and provides read-write view into struct, zero-copy. Two "unsafe" should hint enough that it may break badly.

Answer 6

You should use a bytes buffer instead of a string, the other suggested methods create a SHA1 of variable length, the SHA1 standard length must be 20 bytes (160 bits)

package main

import (
    "crypto/sha1"
    "fmt"
    "encoding/binary"
    "bytes"
)

type myStruct struct {
    ID   string
    Data string
}

func main() {
    var bin_buf bytes.Buffer
    x := myStruct{"1", "Hello"}
    binary.Write(&bin_buf, binary.BigEndian, x)
    fmt.Printf("% x", sha1.Sum(bin_buf.Bytes()))
}

Try it yourself: http://play.golang.org/p/8YuM6VIlLV

It's a really easy method and it works great.

Answer 7

package main

import (
    "crypto/sha1"
    "fmt"
    "encoding/binary"
    "bytes"
)

type myStruct struct {
    ID   [10]byte
    Data [10]byte
}

func main() {
    var bin_buf bytes.Buffer
    x := myStruct{"1", "Hello"}
    binary.Write(&bin_buf, binary.BigEndian, x)
    fmt.Printf("% x", sha1.Sum(bin_buf.Bytes()))
}

binary.Write takes a struct which has fixed length memory allocated datatype.

Answer 8

json.Marshal is the best option to convert a struct to []byte, see example below:

package main

import (
    "encoding/json"
    "fmt"
)

type ExampleConvertToByteArray struct {
    Name    string
    SurName string
}

func main() {

    example := ExampleConvertToByteArray{
        Name:    "James",
        SurName: "Camara",
    }
    
    var exampleBytes []byte
    var err error

    exampleBytes, err := json.Marshal(example)
    if err != nil {
        print(err)
        return
    }

    fmt.Println(string(exampleBytes))
}

Go playground -> https://play.golang.org/p/mnB9Cxy-2H3

Answer 9

Take a look at https://blog.golang.org/go-slices-usage-and-internals Specifically slice internals. The idea is to mimic slice's internal structure and point to our struct instead of a byte sequence:

package main

import (
    "fmt"
    "unsafe"
)

// our structure
type A struct {
    Src int32
    Dst int32
    SrcPort uint16
    DstPort uint16
}

// that is how we mimic a slice
type ByteSliceA struct {
    Addr *A
    Len int
    Cap int
}

func main() {
    // structure with some data
    a := A{0x04030201,0x08070605,0x0A09, 0x0C0B}

    // create a slice structure
    sb := &ByteSliceA{&a, 12, 12} // struct is 12 bytes long, e.g. unsafe.Sizeof(a) is 12

    // take a pointer of our slice mimicking struct and cast *[]byte on it:     
    var byteSlice []byte = *(*[]byte)(unsafe.Pointer(sb))

    fmt.Printf("%v\n", byteSlice)
}

Output:

[1 2 3 4 5 6 7 8 9 10 11 12]

https://play.golang.org/p/Rh_yrscRDV6

Answer 10

Have you considered serializing it to bson? http://labix.org/gobson

Answer 11

var v any
b := (*[unsafe.Sizeof(v)]byte)(unsafe.Pointer(&v))
c := b[:]