6

I was testing my Dataflow pipeline using DirectRunner from my Mac and got lots of "WARNING" message like this, may I know how to get rid of them because it is too much that I can not even see my debug message.

Thanks

Apr 05, 2018 2:14:48 PM org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector verifyUnmodifiedThrowingCheckedExceptions
WARNING: Coder of type class org.apache.beam.sdk.coders.SerializableCoder has a #structuralValue method which does not return true when the encoding of the elements is equal. 
Element com.apigee.analytics.platform.core.service.schema.EventRow@4a590d0b
glytching
  • 44,936
  • 9
  • 114
  • 120
DEWEI SUN
  • 61
  • 1
  • 4

3 Answers3

7

It may help to ensure that all serialized values have proper equals() implementations since SerializableCoder expects them:

The structural value of the object is the object itself. The SerializableCoder should be only used for objects with a proper Object#equals implementation.

You can implement your own Coder for your POJOs. SerializableCoder does not guarantee a deterministic encoding according to docs:

SerializableCoder does not guarantee a deterministic encoding, as Java serialization may produce different binary encodings for two equivalent objects.

This article explains custom coders in details.

antono
  • 978
  • 1
  • 7
  • 18
  • 1
    Do you have any examples or resources on how to implement a Coder? I'm struggling with this same exception and I can't seem to make heads or tails just from the javadoc for the [Coder](https://beam.apache.org/documentation/sdks/javadoc/2.5.0/org/apache/beam/sdk/coders/Coder.html) – nomadic_squirrel Jul 22 '18 at 23:32
  • @nomadic_squirrel I just added good article about custom coder to original answer. – antono Jul 23 '18 at 16:05
  • I'm having these as well. Not sure what the resolution is - should I implement a custom coder? – Arqu Oct 07 '18 at 11:57
1

I had this same problem. I was using SerializableCoder for a class implementing Serializable, and my tests were failing because the PAssert() containsInAnyOrder() method was not using MyClass.equals() to evaluate object equality. The signature of my equals() method was:

public boolean equals(MyClass other) {...}

All I had to do to fix it was to define equals in terms of Object:

public boolean equals(Object other) {...}

This made the warnings go away, and made the tests pass.

Oscar
  • 141
  • 6
1

Just add https://projectlombok.org/features/EqualsAndHashCode

@EqualsAndHashCode 
public class YourRecord implements Serializable {
veaceslav.kunitki
  • 181
  • 2
  • 9