Error unmarshaling when getting array of struct from redis

Question

I am using redigo to save some structs in redis. The thing is that for the same key I need to append new structs, but when I am trying to recover them I cannot unmarshal to an array.

Ie: (ignoring the errors intentionally)

type ADTO struct {
    Value string
}

func main() {
    pool := redis.Pool{
        Dial: func() (conn redis.Conn, e error) {
            return redis.Dial("tcp", "localhost:6379")
        },
        MaxIdle:   80,
        MaxActive: 12000,
    }

    conn := pool.Get()
    defer conn.Close()
    key := "some-key"
    defer conn.Do("DEL", key)

    a := ADTO{Value: "a"}
    bytes, _ := json.Marshal(a)
    conn.Do("APPEND", key, bytes)

    b := ADTO{Value: "b"}
    bytes, _ = json.Marshal(b)
    conn.Do("APPEND", key, bytes)

    c := ADTO{Value: "c"}
    bytes, _ = json.Marshal(c)
    conn.Do("APPEND", key, bytes)

    bytes, _ = redis.Bytes(conn.Do("GET", key))

    adtos := make([]ADTO, 0)

    // the following does not work
    if err := json.Unmarshal(bytes, &adtos); err != nil {
        return
    }
}

If I only append a single struct and retrieving it, then it is working fine

I have tried with redis.ByteSlices with no luck

The redis.ByteSlices function will work as expected if a [list](https://redis.io/topics/data-types#lists) is used instead of a string. Get all values with `LRANGE key 0 -1`. — Charlie Tumahai, Sep 03 '19 at 19:35

score 1 · Accepted Answer · edited Sep 03 '19 at 19:36

1

APPEND will only append to a string, it will not make a JSON array. After first insert, you'll have

{"Value":"a"}

Then after the second one, you'll have

{"Value":"a"}{"Value":"b"}

That's not a JSON array.

You can try using json.Decoder, and do something like:

b, _ = redis.Bytes(conn.Do("GET", key))
dec := json.NewDecoder(bytes.NewReader(b))
items := []ADTO{}
var x ADTO
for dec.Decode(&x) == nil {
  items = append(items, x)
}

edited Sep 03 '19 at 19:36

answered Sep 03 '19 at 18:50

Burak Serdar

46,455
3
40
59

good catch, thank you! The only thing is to use a io reader from the `[]byte` for the `json.NewDecoder`. As mentioned here https://stackoverflow.com/questions/29746123/convert-byte-slice-to-io-reader – hosseio Sep 03 '19 at 18:58

Error unmarshaling when getting array of struct from redis

1 Answers1