Retrieving Msgpack from Http request in java

Question

I am trying to retrieve a msgpack object sent by my rails server in an AsyncTask.

Rails controller - Rails

respond_to do |format|
      format.json { render json: data.to_json }
      format.msgpack { render msgpack: data.to_msgpack }
end

AsyncTask - Java

        HttpEntity messageEntity = httpResponse.getEntity();

            if (messageEntity != null) {
                MessagePack msgpack = new MessagePack();

                ByteArrayOutputStream baos = new ByteArrayOutputStream();
                messageEntity.writeTo(baos);
                byte[] bytes = baos.toByteArray();

                msgpack.read(bytes);    

                [...]
            }

My object is like the following one

{
    "aspects": {
        "1438166757890": {
            "deleted": "0",
            "name": "Choc",
            "updated": "1438166757889"
        },
    [...]
    },
    "items": {
        "1438166757872": {
            "aspects": [
                1438166757890,
                1438166757905,
                1438166757929,
                1438166757947,
                1438166757964,
                1438166757980,
                1438166757997
            ],
            "deleted": "0",
            "name": "OTHER",
            "updated": "1438166757872"
        },
    [...]
    }
}

But when I am trying to read it with my msgpack object like this

Value v = msgpack.read(bytes);

I have this error

java.io.IOException: Invalid byte: -39

Or when I am using those kind of templates

Map<String, Map<Integer, Map<String, String>>> v = msgpack.read(bytes, k);

I am getting

org.msgpack.MessageTypeException: Unexpected array value or org.msgpack.MessageTypeException: Unexpected map value

I did not find any good documentation on Stack or Google concerning this way to read. The official Msgpack website serialize and deserialize a List<String>

List<String> src = new ArrayList<String>();
src.add("msgpack");
src.add("kumofs");
src.add("viver");

MessagePack msgpack = new MessagePack();

byte[] raw = msgpack.write(src);

Value dynamic = msgpack.read(raw);
List<String> dst2 = new Converter(dynamic)
    .read(Templates.tList(Templates.TString));

Did someone already had this problem and if so how to manage Msgpack templates in java ?

Edit :

Stacktrace when I am using msgpack.read(bytes); without templates :

W/System.err( 2999): java.io.IOException: Invalid byte: -39
W/System.err( 2999): at org.msgpack.unpacker.MessagePackUnpacker.readOneWithoutStackLarge(MessagePackUnpacker.java:323)
W/System.err( 2999): at org.msgpack.unpacker.MessagePackUnpacker.readOneWithoutStack(MessagePackUnpacker.java:139)
W/System.err( 2999): at org.msgpack.unpacker.MessagePackUnpacker.readOne(MessagePackUnpacker.java:73)
W/System.err( 2999): at org.msgpack.unpacker.MessagePackUnpacker.readValue(MessagePackUnpacker.java:559)
W/System.err( 2999): at org.msgpack.unpacker.AbstractUnpacker.readValue(AbstractUnpacker.java:65)
W/System.err( 2999): at org.msgpack.MessagePack.read(MessagePack.java:297)
W/System.err( 2999): at org.msgpack.MessagePack.read(MessagePack.java:283)

And when I am using

Template<Map<String, String>> map2 = Templates.tMap(Templates.TString, Templates.TString);

Map<String, String> v = msgpack.read(bytes, map2);

Stacktrace :

W/System.err( 1510): org.msgpack.MessageTypeException: Unexpected array value
W/System.err( 1510): at org.msgpack.unpacker.Accept.acceptArray(Accept.java:79)
W/System.err( 1510): at org.msgpack.unpacker.MessagePackUnpacker.readOneWithoutStackLarge(MessagePackUnpacker.java:264)
W/System.err( 1510): at org.msgpack.unpacker.MessagePackUnpacker.readOneWithoutStack(MessagePackUnpacker.java:139)
W/System.err( 1510): at org.msgpack.unpacker.MessagePackUnpacker.readOne(MessagePackUnpacker.java:73)
W/System.err( 1510): at org.msgpack.unpacker.MessagePackUnpacker.readString(MessagePackUnpacker.java:472)
W/System.err( 1510): at org.msgpack.template.StringTemplate.read(StringTemplate.java:46)
W/System.err( 1510): at org.msgpack.template.StringTemplate.read(StringTemplate.java:25)
W/System.err( 1510): at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31)
W/System.err( 1510): at org.msgpack.template.MapTemplate.read(MapTemplate.java:72)
W/System.err( 1510): at org.msgpack.template.MapTemplate.read(MapTemplate.java:27)
W/System.err( 1510): at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31)
W/System.err( 1510): at org.msgpack.template.MapTemplate.read(MapTemplate.java:72)
W/System.err( 1510): at org.msgpack.template.MapTemplate.read(MapTemplate.java:27)
W/System.err( 1510): at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31)
W/System.err( 1510): at org.msgpack.template.MapTemplate.read(MapTemplate.java:72)
W/System.err( 1510): at org.msgpack.template.MapTemplate.read(MapTemplate.java:27)
W/System.err( 1510): at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31)
W/System.err( 1510): at org.msgpack.MessagePack.read(MessagePack.java:510)
W/System.err( 1510): at org.msgpack.MessagePack.read(MessagePack.java:479)

Could you please post stacktraces of the exceptions? – naXa stands with Ukraine Aug 06 '15 at 14:17 — naXa stands with Ukraine, Aug 06 '15 at 14:17

score 3 · Accepted Answer · edited Aug 10 '15 at 09:43

Thanks to @naXa analysis, I decided to find another msgpack repo on git. After some research, I found MessagePack for Java.

Files extracted, I used sbt to build my .jar files from there with the command ./sbt publish local.

Once build finished, I had main, source and javadoc .jar files into the following dir : ~/msgpack-java-07-develop/msgpack-core/target.

I imported all my .jar files in my Eclipse project, changed the build path and then, to retrieve my msgpack content, I could wrote those simple lines to get my Map containing my values :

HttpEntity messageEntity = httpResponse.getEntity();

if (messageEntity != null) {
    MessagePack msgpack = new MessagePack();

    MessageUnpacker u = msgpack.newUnpacker(messageEntity.getContent());
    MapValue mv = (MapValue) u.unpackValue();

    Map<Value, Value> map = mv.map();

Can you please let me know the version of the message pack library you have used. I am working on a gradle based project and have used 0.8.11 version. new MessagePack(); gives error saying that this constructor is private. — Vaibhav Sharma, Nov 22 '16 at 11:06

score 1 · Answer 2 · answered Aug 06 '15 at 15:07

Not an answer! Just a little analysis.

Here is the class and the code from where the IOException is thrown

default:
    // System.out.println("unknown b "+(b&0xff));
    // headByte = CS_INVALID
    headByte = REQUIRE_TO_READ_HEAD;
    throw new IOException("Invalid byte: " + b); // TODO error FormatException
}

My guess is that msgpack for Android is outdated and doesn't know a type with code -39 (-0x27). Check the differences in types between msgpack for Android and msgpack that you use on server side.

A - in -39 looks suspicious too..

Retrieving Msgpack from Http request in java

2 Answers2