0

I have the following scenario for understanding my problem.

  • There is one external node (Node A), one client App (App1) and one Hazelcast-Jet Job App (Node B, App2).
  • App1 collects data from FlatBuffers and wraps it in an Object HzData. HzData is DataSerializable implemented.
  • Then HzData is put in an IMAP as IMAP calling it hzMap.
  • First, I start my Node A, and then App1. App1 deposits hzMap in Node A.
  • Then, I run App2, which starts in turn starts node B and runs the Jet Job.
  • Little bit information about App2, I have the same HzData in App2 under the same package name.I am adding all the classes related to the job in JobConfig.
  • And then my pipeline consists something similar to this code.

BatchSource<Map.Entry<Integer, HzData>> dataBatchSource = Sources.map('hzMap'); BatchStage<HzData> dataBatchStage = pipe.drawFrom(dataBatchSource ). mapUsingContext(ContextFactories.replicatedMapContext ('hzMap'), (map, data) -> data.getValue()); dataBatchStage.drainTo(Sinks.logger()) The above quote works perfectly fine logging all the data. If I am using a filter like below, it is causing me problems

dataBatchStage.filter(v -> v.getCheck() == 0).drainTo(Sinks.logger());

The above is causing me an error like,

com.hazelcast.jet.JetException: Exception in ProcessorTasklet{filter#24}: java.lang.ClassCastException: com.nexus.api.portables.HzData cannot be cast to com.nexus.api.portables.HzData

The error seems like a Deserialization error, but I wonder how the above logger worked.

I have also tried using filterUsingContext() but still got the same result.

Thanks in advance. Looking forward for your valuable feedback and solutions.

Krishna Nadiminti
  • 1
  • 2
  • seems like the class is being loaded by two different classloaders perhaps? Is it on the classpath and also submitted with the job? – Can Gencer Jun 22 '18 at 20:46
  • I have added the classes to the JobConfig, which I believe they are submitted with the job. Also, the problem persists when loaded with the same class loaders or different class loaders. Interesting thing is my job is set to an infinite loop, the filter seems to work for nearly two iterations ( I can see them in the logs) and suddenly getting this exception. When I am not setting filter loggers can happily see the plain data. – Krishna Nadiminti Jun 22 '18 at 21:08
  • Does it still happen if you don't add the classes to JobConfig but have them on the classpath instead? – Can Gencer Jun 25 '18 at 12:39
  • 1
    Hi Gencer, Thanks for your support. The particular issue has been caused by garbage data from the source. Also, when the supportive classes are in the class path and classes not added to the jobconfig, I would get exception. However, the issue has been resolved now with classes added to the job config for both the apps (App1 and App2). – Krishna Nadiminti Jun 25 '18 at 18:45

0 Answers0