3

Redis is a data structure store but still its recommended to use message-pack (or protobuf) to serialize/deserialize data. I am kind of confuse with Messagepack on top of Redis if data chunks written to Redis is not very big.

Since, Messagepack would need packing and unpacking data as per its own protocol and for sure it will incur some cost and packed data would be store only as "string" data type on Redis.

To leverage on Redis as data structure server a thin layer can be written to read/write directly to/from redis data structure let say between C++ and Python then where exactly message-packs fits in?

Can somebody shed some light on message-pack in context of redis?

Regards,
Rahul

Disclaimer - No offence to Messagepack capability, I know its really awesome :-)

ryadav
  • 456
  • 5
  • 11
  • I use MessagePack as the key of redis. It is contained from name:string and id:64 bit unsigned integer. In this case, I use MessagePack ARRAY https://github.com/msgpack/msgpack/blob/master/spec.md#array-format-family of str https://github.com/msgpack/msgpack/blob/master/spec.md#str-format-family and integer https://github.com/msgpack/msgpack/blob/master/spec.md#int-format-family . I use lua cmsgpack function https://redis.io/commands/eval#cmsgpack via EVAL command on redis. The pros is smaller size than string for id. And easy to convert back and forth. – Takatoshi Kondo Apr 30 '18 at 12:51

1 Answers1

9

There is no single answer, but I can offer a few guidelines.

  • Redis' basic data type is the string - it is binary-safe and can hold up to 0.5GB (probably more in an upcoming version).
  • Key names are strings, but you usually want to a) keep em short and b) they are the only way to access your data so hopefully legible and reconstructable.
  • Values can be strings. If the payload is already a string - no need to serialize/deserialize, just store as is. Common example: jpg or png files.
  • If your app is already using msgpack (or json or protobuff or...), you can store the serialized form.
  • Redis' Lua has built-in libs for dealing with json and msgpack.
  • There are modules (e.g. http://rejson.io) that can extend that.

I hope that helps.

Disclaimer: author of mentioned module, Redis geek and has a black belt working w/ Redis' Lua ;)

Itamar Haber
  • 47,336
  • 7
  • 91
  • 117
  • Thanks! This answers for my current use-case where i have to deal with values which are strings so I would skip serialize/deserialize step messagepack ) for now. – ryadav May 01 '18 at 07:38