1

Using Apache Flink 1.4.2 I get following exception

com.esotericsoftware.kryo.KryoException: java.lang.UnsupportedOperationException
Serialization trace:
data (org.apache.flink.api.java.utils.ParameterTool)
parameterTool (com.test.event.configuration.JobConfiguration)
configuration (com.test.event.steps.PasswordChangedStep)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
    at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:249)
    at org.apache.flink.contrib.streaming.state.RocksDBValueState.value(RocksDBValueState.java:83)
    at com.test.event.transform.StateProcessFunction.processElement(StateProcessFunction.java:39)
    at com.test.event.transform.StateProcessFunction.processElement(StateProcessFunction.java:1)
    at org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:94)
    at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:207)
    at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:69)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:264)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsupportedOperationException
    at java.util.Collections$UnmodifiableMap.put(Collections.java:1457)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:144)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:21)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
    ... 18 more

Not sure what this means. Anyone else getting same behaviour when using the process function (https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/stream/operators/process_function.html)?

razvan
  • 559
  • 7
  • 23

1 Answers1

2

Turns out the cause was due to using the ParameterTool as part of the state, we took it out and only kept the part of configuration that was required (a few int values) and it works now

razvan
  • 559
  • 7
  • 23