Spring Cloud Stream - Adding fasterxml ObjectNode to trusted package of kafka

Question

I am using Spring-cloud-stream version 3.0.4

I am writing a JSON consolidator, which listens to multiple streams, store the JSONs in state-stores and then consolidates it to produce an output JSON. Since my service is just a JSON aggregator I dont want to convert JSONS into Java Objects. So what I am trying to do is publish JSONs from upstream services as JsonNode. Do the merging and publish it on the downstream topic.

To add JsonNode as a trusted package I have declared a bean in my Application Class like below.

@Bean
public KafkaHeaderMapper customKafkaHeaderMapper()
{
    DefaultKafkaHeaderMapper kafkaHeaderMapper = new DefaultKafkaHeaderMapper();
    kafkaHeaderMapper.addTrustedPackages("com.fasterxml.jackson.databind.node");
    return kafkaHeaderMapper;

}

and added the below entry in my application.yml

spring.application.name: stream-aggregator
spring.cloud.stream.bindings.formDataIn:
  destination: form-data
  contentType: application/json
spring.cloud.stream.kafka.streams.binder:
  configuration:
    default.key.serde: org.springframework.kafka.support.serializer.JsonSerde
    default.value.serde: org.springframework.kafka.support.serializer.JsonSerde
    spring.json.key.default.type: com.company.datamapper.domain.EmpUUID
    spring.json.value.default.type: com.fasterxml.jackson.databind.JsonNode
    commit.interval.ms: 1000
spring.cloud.stream.kafka.binder.headerMapperBeanName:
  customKafkaHeaderMapper

However, this config is not working and I am getting the following error.

Caused by: java.lang.IllegalArgumentException: The class 'com.fasterxml.jackson.databind.node.ObjectNode' is not in the trusted packages: [java.util, java.lang, com.fasterxml.jackson.databind]. If you believe this class is safe to deserialize, please provide its name. If the serialization is only done by a trusted source, you can also enable trust all (*).
    at org.springframework.kafka.support.converter.DefaultJackson2JavaTypeMapper.getClassIdType(DefaultJackson2JavaTypeMapper.java:125) ~[spring-kafka-2.3.7.RELEASE.jar:2.3.7.RELEASE]
    at org.springframework.kafka.support.converter.DefaultJackson2JavaTypeMapper.toJavaType(DefaultJackson2JavaTypeMapper.java:99) ~[spring-kafka-2.3.7.RELEASE.jar:2.3.7.RELEASE]
    at org.springframework.kafka.support.serializer.JsonDeserializer.deserialize(JsonDeserializer.java:425) ~[spring-kafka-2.3.7.RELEASE.jar:2.3.7.RELEASE]
    at org.apache.kafka.streams.processor.internals.SourceNode.deserializeValue(SourceNode.java:60) ~[kafka-streams-2.3.1.jar:?]
    at org.apache.kafka.streams.processor.internals.RecordDeserializer.deserialize(RecordDeserializer.java:66) ~[kafka-streams-2.3.1.jar:?]

I have 2 Queries

How to fix above error.
Is my approach optimal orI should be doing the aggregation differently.

Double check your configuration; according to the error, you are missing the final `.node` `>trusted packages: [java.util, java.lang, com.fasterxml.jackson.databind]`. — Gary Russell, May 09 '20 at 20:41
Thanks, Gary for the reply. However, I copy-pasted the customKafkaHeaderMapper definition from my workspace :( — Anupam Gupta, May 09 '20 at 21:20

Gary Russell · Answer 1 · 2020-05-10T00:13:34.387

The problem is in the deserializer, not the header mapper...

at org.springframework.kafka.support.serializer.JsonDeserializer.deserialize(JsonDeserializer.java:425)

spring.json.value.default.type: com.fasterxml.jackson.databind.JsonNode

We will automatically "trust" the package for the default type but sub-packages are not trusted.

You need to add the ...node package to

spring.json.trusted.packages

in the binder configs.

We should probably support trusting sub packages; feel free to open an issue on GitHub against spring-kafka.

Spring Cloud Stream - Adding fasterxml ObjectNode to trusted package of kafka

1 Answers1