Golang + postgres storing gob data

Question

I'm trying to store encoded data using encoding/gob from Golang into Postgres. I'm using Gorm as well.

First, sending form data using

if err := json.NewDecoder(r.Body).Decode(model); err != nil {
  http.Error(w, err.Error(), http.StatusBadRequest)
  return
}

Currently client_encoding is set to UTF8 in the postgres database. Here's what I'm using to encode:

type payload struct {
    Key     []byte
    Nonce   *[nonceLength]byte
    Message []byte
}

// Encrypt message
p.Message = secretbox.Seal(p.Message, plaintext, p.Nonce, key) // key set prior to this

buf := &bytes.Buffer{}
if err := gob.NewEncoder(buf).Encode(p); err != nil {
    return nil, err
}

return buf.Bytes(), nil

Then I store string(buf.Bytes()) which is stored in the database column which is currently a string type. Now I'm a novice with encoding, and I think gob just has a different encoding for my database. I'm receiving this error in console:

(pq: invalid byte sequence for encoding "UTF8": 0xff)

I've been following this gist for encryption/decryption: https://gist.github.com/fuzzyami/f3a7231037166117a6fef9607960aee7

From what I've read, I shouldn't be encoding structs into the db, in this case p, unless using gob. Correct me if I'm wrong with that (can't find the resource at the moment where I found this).

Is anyone able to point me in the right direction for storing this data in Postgres which is decrypted later? I'm clearly not understanding the encoding process, and not entirely sure where to start out here with reading resources, so any help is appreciated!

score 0 · Answer 1 · answered Nov 11 '18 at 18:01

0

Use a bytea column in postgresql to store a []byte, and skip the conversion to string.

answered Nov 11 '18 at 18:01

hobbs

223,387
19
210
288

I've just tried to implement this, but when coding the body into the struct I get `json: cannot unmarshal string into Go struct field Table.Column of type pq.ByteaArray`. Any thoughts on getting around this? Seems like this can get a bit messy. – Greg Miller Nov 11 '18 at 18:35
That's coming from code that isn't in your question, and you're doing something very wrong. – hobbs Nov 11 '18 at 18:38
Just passing form data and decoding into a struct. I'll update the question now. – Greg Miller Nov 11 '18 at 18:42

score 0 · Answer 2 · answered Nov 11 '18 at 20:58

0

Took a look at https://golang.org/src/encoding/base64/example_test.go

Was able to use

return base64.StdEncoding.EncodeToString(buf.Bytes()), nil

Which successfully stored in the database.

answered Nov 11 '18 at 20:58

Greg Miller

1,064
13
22

Golang + postgres storing gob data

2 Answers2