Deserializing nested objects in Jackson

Question

This should be straight forward based on the docs and examples I've seen around but I haven't managed to get this working. I apologize if this has been posted elsewhere but I have not been able to find the same issue even though it relates to nested object deserialization with the Jackson library.

Key Items:

Jackson parser is trying to map nested objects into a LinkedHashMap instead of the actual object/class type
Using 3rd party class so no annotations possible
The JSON data is not modifiable as it's being generated elsewhere

Basically, my JSON is being generated from GCP logs and looks like this (simplified for readability):

{
  "logName": "projects/my_project_id/logs/stdout",
  "severity": "ERROR",
  "timestamp": "2021-07-25T01:32:56.371417530Z",
  "receiveTimestamp": "2021-07-25T01:33:01.118327598Z",
  "jsonPayload": {
    "message": "application log message"
  },
  "resource": {
    "type": "gce_instance",
    "labels": {
      "project_id": "my_project_id",
      "instance_id": "my_instance_id",
      "zone": "my_zone"
    }
  },
  "operation": {
    "first": true,
    "id": "<operation_id>",
    "producer": "producer_name"
  }
}

Using the Jackson databind libraries I'm trying to parse and deserialise this into Java objects.

As I'm working with GCP, Google provides libraries for these log objects. Namely com.google.api.services.logging.v2.model.LogEntry and it's related objects like HttpRequest, LogEntryOperation, etc. This object mostly has simple objects like String, Integer and Maps however there are some which are objects mentioned above (HttpRequest, LogEntryOperation, etc).

e.g

public final class LogEntry extends GenericJson {

    private HttpRequest httpRequest;

    private String insertId;

    private Map<String, Object> jsonPayload;

    private Map<String, String> labels;

    private String logName;

    private MonitoredResourceMetadata metadata;

    private LogEntryOperation operation;

    ...
}

From the docs and examples I've seen it should be pretty straight forward to use the readValue method and pass it the class I want to map to like so:

byte[] data = jsonString.getBytes();
ObjectMapper mapper = new ObjectMapper();
mapper.readValue(new ByteArrayInputStream(data), LogEntry.class);

However I'm getting this error:

com.fasterxml.jackson.databind.JsonMappingException: Can not set com.google.api.services.logging.v2.model.LogEntryOperation field com.google.api.services.logging.v2.model.LogEntry.operation to java.util.LinkedHashMap (through reference chain: com.google.api.services.logging.v2.model.LogEntry["operation"])

This occurs for each nested object within the LogEntry class. Just to confirm I cloned the LogEntry class and updated those object, say LogEntryOperation, to a Map and it's able to work.

Now the problem is I don't want to be cloning this class given it's part of a library and I can't change the data in GCP either.

I also attempted to use config (SerializationFeature and DeserializationFeature) for the ObjectMapper but I couldn't find anything that could help. I assumed the Jackson parser was able to determine the nested objects when you pass the class as an argument.

Am I missing a config or something? Or is this not possible using the Jackson parser?

Thanks!

Edit 1: The complete stack trace:

com.abc.logging.data.parsing.LogEntryParser$LogParseException: Failed to parse class com.google.api.services.logging.v2.model.LogEntry value from json payload
    at com.abc.logging.data.parsing.LogEntryParser.parse(LogEntryParser.java:87)
    at com.abc.logging.data.parsing.LogEntryParser.parse(LogEntryParser.java:34)
    at com.abc.logging.data.parsing.LogEntryParserTest.testMapper(LogEntryParserTest.java:191)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not set com.google.api.services.logging.v2.model.LogEntryOperation field com.google.api.services.logging.v2.model.LogEntry.operation to java.util.LinkedHashMap (through reference chain: com.google.api.services.logging.v2.model.LogEntry["operation"])
    at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:397)
    at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:356)
    at com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase.wrapAndThrow(ContainerDeserializerBase.java:181)
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:552)
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:377)
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4524)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3503)
    at com.abc.logging.data.parsing.LogEntryParser.parse(LogEntryParser.java:81)
    ... 28 more
Caused by: java.lang.IllegalArgumentException: Can not set com.google.api.services.logging.v2.model.LogEntryOperation field com.google.api.services.logging.v2.model.LogEntry.operation to java.util.LinkedHashMap
    at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:167)
    at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:171)
    at java.base/jdk.internal.reflect.UnsafeObjectFieldAccessorImpl.set(UnsafeObjectFieldAccessorImpl.java:81)
    at java.base/java.lang.reflect.Field.set(Field.java:780)
    at com.google.api.client.util.FieldInfo.setFieldValue(FieldInfo.java:275)
    at com.google.api.client.util.FieldInfo.setValue(FieldInfo.java:231)
    at com.google.api.client.util.GenericData.put(GenericData.java:98)
    at com.google.api.client.util.GenericData.put(GenericData.java:43)
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:547)
    ... 33 more
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.583 s <<< FAILURE! - in com.abc.logging.data.parsing.LogEntryParserTest
[ERROR] testMapper(com.abc.logging.data.parsing.LogEntryParserTest)  Time elapsed: 0.904 s  <<< FAILURE!
java.lang.AssertionError
    at com.abc.logging.data.parsing.LogEntryParserTest.testMapper(LogEntryParserTest.java:212)

Edit 2: I attempted to use Mixin with @JsonDeserialize and a separate deserialization class just for these objects however it doesn't seem to be calling deserialization class. It still fails with the same exception.

The Mixin class:

public abstract class LogEntryOperationMixin {
    @JsonDeserialize(using = MyLogEntryOpDeserializer.class)
    private LogEntryOperation operation;
}

The deserialization class:

public class MyLogEntryOpDeserializer extends JsonDeserializer<LogEntryOperation> {

    @Override
    public LogEntryOperation deserialize(JsonParser parser, DeserializationContext deserializer) {
        LogEntryOperation entryOperation = new LogEntryOperation();
        // ... populate with data
        return entryOperation;
    }
}

My main class:

ObjectMapper mapper = new ObjectMapper();
mapper.addMixIn(LogEntry.class, LogEntryOperationMixin.class);
mapper.readValue(jsonData, LogEntry.class);

Have you tried using `gson` instead of `jackson`? `LogEntryOperation` already implements Map.class. Can you post the complete stacktrace? — papaya, Jul 26 '21 at 11:12
@papaya Thanks for the response. I haven't attempted gson just yet as the current framework relies in jackson for more then just the component I'm working on and I wanted minimal impact with my changes. However I'll take a look if that's the only way to go. I also added the entire stack trace. — abitofeverything, Jul 26 '21 at 23:28
Does this answer your question? [Jackson - How to process (deserialize) nested JSON?](https://stackoverflow.com/questions/11747370/jackson-how-to-process-deserialize-nested-json) — Martin Zeitler, Jul 26 '21 at 23:54
[`LogEntryOperation`](https://developers.google.com/resources/api-libraries/documentation/logging/v2/java/latest/com/google/api/services/logging/v2/model/LogEntryOperation.html) is clear about it: "Additional information about a potentially long-running operation with which a log entry is associated." ...this means, that it may eventually be absent. — Martin Zeitler, Jul 27 '21 at 00:01
Thanks @MartinZeitler, I assumed I shouldn't need to write wrapper classes as LogEntry is an established class from a library. And I pass it as a argument to readValue which I figured it could determine the mapping without annotations. And yes LogEntryOperation is not always present but that's just the example exception for this one object. It occurs for all non-basic objects such as HttpRequest, MonitoredResource, MonitoredResourceMetadata, etc. So if the data doesn't have operation (LogEntryOperation) but has say httpRequest (HttpRequest) it will fail with the same exception. — abitofeverything, Jul 27 '21 at 01:51
It appears there is a problem using the Jackson parser with GenericJson and it's @Key annotation used by Google. I created a identical class without the inheritance from GenericJson and was able to map without issues. I'll have to look further into what GenericJson is doing to the parsing. — abitofeverything, Jul 27 '21 at 13:03

score 0 · Answer 1 · answered Jul 28 '21 at 01:39

So I was hunting down the wrong path with determining how to do this with Jackson out of the box so to speak (e.g. without customization). The google-api-client objects extend GenericJson and together with the @Key annotations, Jackson is not able to deserialize subclasses and attempts to default them to LinkedHashMaps.

One method is to use either a clone or a wrapper and parse these subclasses manually (via mixin and the @JsonDeserialize annotation). Then map these back to the original Google object. It's not elegant but it gets the job done.

Deserializing nested objects in Jackson

1 Answers1