How to customly serialize or convert a Map property with custom key type in Jackson JSON (de-)serialization?

Question

I'm serializing instances of

@JsonIdentityInfo(
    generator = ObjectIdGenerators.PropertyGenerator.class,
    property = "id",
    scope=Entity1.class)
public class Entity1 {
    private Long id;
    @JsonSerialize(converter = ValueMapListConverter.class)
    @JsonDeserialize(converter = ValueMapMapConverter.class)
    private Map<Entity2, Integer> valueMap = new HashMap<>();

    public Entity1() {
    }

    public Entity1(Long id) {
        this.id = id;
    }

    [getter and setter]
}

and

@JsonIdentityInfo(
    generator = ObjectIdGenerators.PropertyGenerator.class,
    property = "id",
    scope=Entity2.class)
public class Entity2 {
    private Long id;

    public Entity2() {
    }

    public Entity2(Long id) {
        this.id = id;
    }

    [getter and setter]
}

with

ObjectMapper objectMapper = new ObjectMapper();
Entity1 entity1 = new Entity1(1l);
Entity2 entity2 = new Entity2(2l);
entity1.getValueMap().put(entity2, 10);
String serialized = objectMapper.writeValueAsString(entity1);
Entity1 deserialized = objectMapper.readValue(serialized, Entity1.class);
assertEquals(entity1,
        deserialized);

@JsonSerialize and @JsonDeserialize have been added in order to be able to serialize the map with complex key type. The converters are

public class ValueMapMapConverter extends StdConverter<List<Entry<Entity2, Integer>>, Map<Entity2, Integer>> {

    @Override
    public Map<Entity2, Integer> convert(List<Entry<Entity2, Integer>> value) {
        Map<Entity2, Integer> retValue = new HashMap<>();
        for(Entry<Entity2, Integer> entry : value) {
            retValue.put(entry.getKey(), entry.getValue());
        }
        return retValue;
    }
}

and

public class ValueMapListConverter extends StdConverter<Map<Entity2, Integer>, List<Entry<Entity2, Integer>>> {

    @Override
    public List<Entry<Entity2, Integer>> convert(Map<Entity2, Integer> value) {
        return new LinkedList<>(value.entrySet());
    }
}

However, the annotations have no effect since the deserialization still fails due to

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot find a (Map) Key deserializer for type [simple type, class richtercloud.jackson.map.custom.serializer.Entity2]
 at [Source: (String)"{"id":1,"valueMap":{"richtercloud.jackson.map.custom.serializer.Entity2@bb":10}}"; line: 1, column: 1]
 at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
 at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1451)
 at com.fasterxml.jackson.databind.deser.DeserializerCache._handleUnknownKeyDeserializer(DeserializerCache.java:589)
 at com.fasterxml.jackson.databind.deser.DeserializerCache.findKeyDeserializer(DeserializerCache.java:168)
 at com.fasterxml.jackson.databind.DeserializationContext.findKeyDeserializer(DeserializationContext.java:500)
 at com.fasterxml.jackson.databind.deser.std.MapDeserializer.createContextual(MapDeserializer.java:248)
 at com.fasterxml.jackson.databind.DeserializationContext.handlePrimaryContextualization(DeserializationContext.java:651)
 at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:471)
 at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
 at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
 at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
 at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:477)
 at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4178)
 at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3997)
 at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2992)
 at richtercloud.jackson.map.custom.serializer.TheTest.testSerialization(TheTest.java:29)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
 at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
 at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
 at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
 at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
 at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
 at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
 at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
 at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

I took a look into map serialization and am pretty sure I understood the basic concept and expect a key serializer to be unnecessary because the conversion takes place first and the converted output is a list which doesn't need one.

There might be further issues with the serialization of Entry which I will then overcome by using a different class, eventually my own.

A SSCCE can be found at https://gitlab.com/krichter/jackson-map-custom-serializer.

I'm using Jackson 2.9.4.

Upstream issue to vote for to avoid the need for custom deserializers: https://github.com/FasterXML/jackson-databind/issues/3984 — Geoffrey De Smet, Jun 14 '23 at 06:32

Jose Da Silva Gomes · Accepted Answer · 2018-03-11T03:00:33.240

The problem here is that when you use Map.Entry the key has to be a string, because it gets serialized like {"key": value}.

You have two options

Your first option if you can serialize your object as string you can use it as the json key.

This is posible in two cases, when the object contains a single field (like the one in your example). e.g.

new SingleFieldObject(2l) // can be serialized as "2"

Or when constains multiple fields that can be represented as string. e.g.

new MultipleFieldObject("John", 23) // can be serialized as "John 23 Years Old"

Now that the custom object can be represented as string you could use either a map or a list of entries.

To use a simple map just use the attribute 'keyUsing' in the annotations, and also you have to define the custom serializer and deserializer.

public class MyKeyDeserializer extends KeyDeserializer {
    @Override
    public Entity2 deserializeKey(String key,
                                  DeserializationContext ctxt) throws IOException {
        return new Entity2(Long.parseLong(key));
    }
}

public class MyKeySerializer extends JsonSerializer<Entity2> {
    @Override
    public void serialize(Entity2 value,
                          JsonGenerator gen,
                          SerializerProvider serializers) throws IOException {
        gen.writeFieldName(value.getId().toString());
    }
}

Then you annotate the field with your serializer and deserializer:

@JsonSerialize(keyUsing = MyKeySerializer.class) // no need of converter
@JsonDeserialize(keyUsing = MyKeyDeserializer.class) // no need of converter
private Map<Entity2, Integer> valueMap = new HashMap<>();

Using this object.

Entity1 entity1 = new Entity1(1l);
Entity2 entity2_1 = new Entity2(2l);
Entity2 entity2_2 = new Entity2(3l);
entity1.getValueMap().put(entity2_1, 21);
entity1.getValueMap().put(entity2_2, 22);

A JSON like this is generated

{
    "id": 1,
    "valueMap": {
        "2": 21,
        "3": 22
    }
}

To use a list you could use the converters in your example, but instead Entity2 you return a String for the key.

public class ValueMapListConverter 
    extends StdConverter<Map<Entity2, Integer>, List<Entry<String, Integer>>> {
    @Override
    public List<Entry<String, Integer>> convert(Map<Entity2, Integer> value) {
        List<Entry<String, Integer>> result = new ArrayList<>();
        for (Entry<Entity2, Integer> entry : value.entrySet()) {
            result.add(new SimpleEntry<>(entry.getKey().getId().toString(), 
                       entry.getValue()));
        }
        return result;
    }
}

public class ValueMapMapConverter 
    extends StdConverter<List<Entry<String, Integer>>, Map<Entity2, Integer>> {
    @Override
    public Map<Entity2, Integer> convert(List<Entry<String, Integer>> value) {
        Map<Entity2, Integer> retValue = new HashMap<>();
        for(Entry<String, Integer> entry : value) {
            retValue.put(new Entity2(Long.parseLong(entry.getKey())), entry.getValue());
        }
        return retValue;
    }
}

A JSON like this is generated

{
    "id": 1,
    "valueMap": [
        { "2": 21 },
        { "3": 22 }
    ]
}

In both cases the value Integer could be a complex object.

Your second option is to use a custom object, again you have multiple options, one object that hold all the fields of the key and the field/fields of the value.

// ... serialization - deserialization of the object
public class CustomObject {
    private Long id; // ... all key fields
    private int value; // ... all value fields
}

Then you use the converters public class ValueListMapConverter extends StdConverter<List<CustomObject>, Map<Entity2, Integer>> and public class ValueMapMapConverter extends StdConverter<Map<Entity2, Integer>, List<CustomObject>>

This generates a JSON like this

{
    "id": 1,
    "valueMap": [
        { "id": 2, "value": 21 },
        { "id": 3, "value": 22 }
    ]
}

You could use a map instead a list and use a key, and the rest of the fields of the key object, together with the value fields in a custom object.

// ... serialization - deserialization of the object
public class CustomObject {
    // ... rest of the key fields
    private int value; // ... all value fields
}

The converters public class ValueListMapConverter extends StdConverter<Map<Long, CustomObject>, Map<Entity2, Integer>> and public class ValueMapMapConverter extends StdConverter<Map<Entity2, Integer>, Map<Long, CustomObject>>

This generates a JSON like this

{
    "id": 1,
    "valueMap": {
        "2": { "value": 21 },
        "3": { "value": 22 },
    }
}

Thanks for you detailed answer. I tried some converter-based approaches which failed for different reasons which seem insignificant given the fact that afaiu Jackson 2.9 has massive problems with serialization support, e.g. https://github.com/FasterXML/jackson-databind/issues/1419 which leads to failures which are completely unrelated to the problem that some serializations are in fact simply not supported or working, yet. I created a working custom serializer in commit 2ea6797 of the SSCCE (that'd be the third option), although one has to take care about type erasure issues, like described at — Kalle Richter, Mar 11 '18 at 17:25
https://stackoverflow.com/questions/49213606/bypass-runtime-type-erasure-for-generic-map-serializer. I'm accepting your answer because it points out the crucial part "key has to be a string". I'd be happy is you incorporate the third approach and remove the first becuase it's not for arbitrarily complex objects. Do you have an idea how to make the the map serializer use ID references provided by `@JsonIdentityInfo`? — Kalle Richter, Mar 11 '18 at 17:31
Ok i can remove the first approach. But what do you mean with incorporate the third? you mean extend/improve. Also is something specific failing right now?, i see the tests working good. And refering to `@JsonIdentityInfo` at least i'm not aware how the serializer could use the id (if i understand correctly we're talking of `JsonIdentityInfo#property`). Of course you could use reflection and get all the info you want from the annotation, but i'm not sure if that's what you want. — Jose Da Silva Gomes, Mar 11 '18 at 18:44
I didn't test your approaches because my third solution worked well for me, but since questions and answers are for everybody, it's good to have it (and yes, I meant to extend you answer if you want). The second thing reading id references is to have `{f1: {...object1}, f2: [1: {...object2}]}` instead of `{f1: {...object1}, f2: [{...object1}: {...object2}]}` where `1` is the id of `object1`, but that's basically a different question. — Kalle Richter, Mar 11 '18 at 18:50
The solution in the branch 'inheritance' is the second approach option 1. Also that reason that the answer is for everybody applies to the first approach, someone may use it, so i think just extending 2nd approach would be good. As for your json in the comment, i didn't get it, you're using keys in an array, and obviously a json key cannot be an object, but it's out of this scope anyway. — Jose Da Silva Gomes, Mar 11 '18 at 19:27

How to customly serialize or convert a Map property with custom key type in Jackson JSON (de-)serialization?

1 Answers1

Linked