Apache beam with MessagePack - how do I get values from Map?

Question

Inside an Apache Beam transform I can successfully transform pub/sub values (in MessagePack format) that I read from Google Cloud Pub/Sub into a map of MessagePack Value objects like this:

    @ProcessElement
    public void processElement(ProcessContext c) 
    {
        Map<Value, Value> map = MessagePack.newDefaultUnpacker(c.element().getPayload()).unpackValue().asMapValue().map();

When I inspect map I can see the following:

If I then try to get a value like this it always returns null:

map.get("Tz")

How do I get a value? Do I need to transform the values in a different way, or do I need a different way of retrieving them?

score 2 · Answer 1 · answered Nov 07 '18 at 23:43

2

The map is Map, so the key is a value object, but you are using a String s the key when you do map.get("Tz")

Can you create a Value object using Tz and attempt to get based on that Value object?

answered Nov 07 '18 at 23:43

Justin

1,356
2
9
16

Thanks @Justin +1 - I was thinking on these lines. I've posted the solution - I needed an additional import for `ValueFactory`. – Chris Halcrow Nov 07 '18 at 23:51

Chris Halcrow · Answer 2 · 2018-11-12T01:16:07.670

The map keys are MessagePack Value objects, so I needed to do this to reference the key, and return the values as strings:

import org.msgpack.value.ValueFactory;

map.get(ValueFactory.newString("Tz")).toString()

See also In messagepack, error while getting value from MapValue.. Please help me (the solution didn't work in my case, but there are some suggestions there for dealing with different types that can be used in a Map)

Apache beam with MessagePack - how do I get values from Map?

2 Answers2