Questions tagged [msgpack]

MessagePack is a binary-based efficient object serialization library. It enables to exchange structured objects between many languages like JSON. But unlike JSON, it is very fast and small.

MessagePack is a binary-based efficient object serialization library. It enables to exchange structured objects between many languages like JSON. But unlike JSON, it is very fast and small.

Typical small integer (like flags or error code) is saved only in 1 byte, and typical short string only needs 1 byte except the length of the string itself. [1,2,3] (3 elements array) is serialized in 4 bytes using MessagePack as follows:

> require 'msgpack'
> msg = [1,2,3].to_msgpack  #=> "\x93\x01\x02\x03"
> MessagePack.unpack(msg)   #=> [1,2,3]

Scalable RPC

MessagePack-RPC is cross-language RPC library for client, server and cluster applications. Because it releases you from complicated network programming completely and provides well-designed API, you can easily implement advanced network applications with MessagePack-RPC.

require 'msgpack/rpc'
class MyHandler
  def add(x,y) return x+y end
end
svr = MessagePack::RPC::Server.new
svr.listen('0.0.0.0', 18800, MyHandler.new)
svr.run

require 'msgpack/rpc'
c = MessagePack::RPC::Client.new('127.0.0.1',18800)
result = c.call(:add, 1, 2)  #=> 3

MessagePack is available for many languages:

  • C
  • C++
  • D
  • Erlang
  • Go
  • Haskell
  • Java
  • OCaml
  • Python
  • Perl
  • PHP
  • Ruby
  • Scala
409 questions
2
votes
1 answer

Not the same output when using Messagepack encoding on Golang and Rust

I am trying to make communicate a Go server with a Rust client (and vice-versa), and to do so, I want to serialize (or Marshal as you would say in Go) a struct in order to send it. Here are my codes : package main import ( "fmt" …
edode
  • 43
  • 5
2
votes
1 answer

Serde MsgPack versioning

I need to serialize some data into files. For the sake of memory efficiency, I want to use the default compact serializer of MessagePack (MsgPack), as it only serializes field values w/o their names. I also want to be able to make changes to the…
at54321
  • 8,726
  • 26
  • 46
2
votes
0 answers

How to convert nlohmann binary to uint8_t vector?

I get a msgpack object, wrapping an array with uint8_t values from an API and load it to the nlohmann/json library with json::from_msgpack, when I print the json document I get this: { "raw_data": { "bytes": [4, 3, 2, 0, 0, 116, 68], …
Paul Würtz
  • 1,641
  • 3
  • 22
  • 35
2
votes
2 answers

dask-yarn job fails with dumps_msgpack ImportError while reading parquet

I am trying to do a simple read and count of a small parquet file (10K records) using dask-yarn on an AWS EMR cluster with one master and one worker node, both are m5.xlarge instances. I am trying to execute the following code just to test my…
2
votes
1 answer

msgpack dictionary with tuple keys

import msgpack path = 'test.msgpack' with open(path, "wb") as outfile: outfile.write(msgpack.packb({ (1,2): 'str' })) works fine, now with open(path, 'rb') as infile: print(msgpack.unpackb(infile.read())) errors with Traceback (most recent…
theonlygusti
  • 11,032
  • 11
  • 64
  • 119
2
votes
1 answer

Wrong result converting MsgPack timestamp 64 fields to DateTime

I am trying to implement the MsgPack TimeStamp 64 type in C#. This is what the specification says: timestamp 64 stores the number of seconds and nanoseconds that have elapsed since 1970-01-01 00:00:00 UTC in 32-bit unsigned…
Louis Somers
  • 2,560
  • 3
  • 27
  • 57
2
votes
0 answers

M1 Mac Ruby on Rails msgpack

After attempting to use Ruby on Rails on an M1 Mac, I came across the following error. It seems msgpack is getting a loading error related to the M1 Mac's architecture. Has anyone found a work around for…
Papi
  • 255
  • 1
  • 2
  • 13
2
votes
1 answer

.net 5 - Signalr - Messagepack - lz4 - Error: unable to find ext type 98

Got this error un .net core 5 solution with angular client after enabling lz4BlockArray compression. Without compression everithing seems to work pretty fine. Looking around for motivation but i found nothing. Seems like some decoders aren't loaded…
montipirlo
  • 23
  • 2
2
votes
1 answer

msgpack serialization in dask.protocol.core raising TypeError: can not serialize 'Variable' object

I have dask distributed as part of a larger effort with workflow (prefect), and am seeing an error in my dask workers. The worker receives its work, but quickly raises this exception: TypeError: can not serialize 'Variable' object The full trace…
Greg Desmarais
  • 336
  • 3
  • 5
2
votes
2 answers

How to use message pack in xamarin forms

I have to use MessagePack to deserialize data returned from a web API (ASP.net), which was Serialized using the same package. Used the following code public async static Task DeserializeAsync(Stream stream) { if (stream == null) { …
Nikhileshwar
  • 1,666
  • 1
  • 11
  • 26
2
votes
1 answer

MessagePack deserialize exception FormatterNotRegisteredException

I have a unity project where i need to be able to deserialize really fast. I tried MessagePack. I followed the tutorial but when the deserialization is triggered i receive this error: FormatterNotRegisteredException: DeserializedObject is not…
cUser
  • 392
  • 8
  • 25
2
votes
0 answers

trouble in gem 'msgpack'

I have trouble with Heroku. My app succesffuly download on Heroku, but when I try to run rake db:migrate, I get this: rake aborted! LoadError: libruby.so.2.5: cannot open shared object file: No such file or directory -…
DmitrBayan
  • 21
  • 2
2
votes
1 answer

Direct json to StructureMap serialized byte[]

I have the following JSON (received as byte[] from a web socket): { "ev": "T", // Event Type "sym": "MSFT", // Symbol Ticker "x": 4, // Exchange ID "i": "12345", // Trade ID "z": 3,…
morleyc
  • 2,169
  • 10
  • 48
  • 108
2
votes
2 answers

How do you use a custom NeuralCoref model in Spacy?

I followed the guidelines on how to train a a neural coreference model using NeuralCoref. I now have a model, but can not figure out how to use the coref model in Spacy. The following shown in the manual does not describe how to load a custom…
Johan
  • 21
  • 7
2
votes
1 answer

Could not load type MessagePack with Azure signalR

I have been using Microsoft.Azure.SignalR library 1.1.1 which is dependent of MessagePack 1.7.3.4 in my .Net Core 3.0 library. Now, I am trying to use the latest version of MessagePack library (2.0.335). It results me the below error. Thanks in…
George
  • 71
  • 11