0

I am trying to use Kryo in spark for the SAMRecord class, but getting the error shown at runtime. I don't understand what this error means, can anyone explain? What do I need to change for Kryo to work with this? The problem is with the member variable mAlignmentBlocks of type List<AlignmentBlock>. Does Kryo have a problem with such lists?

Note that I have registered both AlignmentBlock and SAMRecord by using the following code:

kryo.register(classOf[AlignmentBlock])
kryo.register(classOf[SAMRecord])

And the error I get is this:

com.esotericsoftware.kryo.KryoException: java.lang.UnsupportedOperationException Serialization trace: mAlignmentBlocks (htsjdk.samtools.SAMRecord) at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.read(FieldSerializer.java:626) at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:221) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729) at com.twitter.chill.Tuple2Serializer.read(TupleSerializers.scala:42) at com.twitter.chill.Tuple2Serializer.read(TupleSerializers.scala:33) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729) at org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:228) at org.apache.spark.serializer.DeserializationStream.readValue(Serializer.scala:171) at org.apache.spark.serializer.DeserializationStream$$anon$2.getNext(Serializer.scala:201) at org.apache.spark.serializer.DeserializationStream$$anon$2.getNext(Serializer.scala:198) at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73) at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371) at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327) at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:32) at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39) at org.apache.spark.util.collection.ExternalAppendOnlyMap.insertAll(ExternalAppendOnlyMap.scala:152) at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:45) at org.apache.spark.shuffle.BlockStoreShuffleReader.read(BlockStoreShuffleReader.scala:89) at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:98) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306) at org.apache.spark.rdd.RDD.iterator(RDD.scala:270) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)

Community
  • 1
  • 1
pythonic
  • 20,589
  • 43
  • 136
  • 219

1 Answers1

-1

you are calling a method on a class that implementing an interface but the implementation of this method is not present.

Tokazio
  • 516
  • 2
  • 18
  • Can you please elaborate. I didn't understand what you said. Implementation of which class is not present? You mean the List class? If so, how can I add that? – pythonic Sep 30 '16 at 17:46
  • i don't know kryo and i now that my explication can be abstract... found the method you call and that is giving you this exception. see its source code. i can't do ot myself as im on my phone... – Tokazio Sep 30 '16 at 17:48
  • seems to be in kryo. sorry i can help you more because don't knowing more about kryo. put the code where you call kryo to serialize – Tokazio Sep 30 '16 at 18:12