0

I refactor my code to work with kryo serialization. Everything works fine except deserialize a property of geomtry from certain class. No exception is thrown (I set "spark.kryo.registrationRequired" to true). On debug I try to collect the data and I see that the data in the geomtry is just empty. As a result I understand that the deserialize was fail. Geomtry is from type of - Any(scala) because it is a complex property maybe. My question is why the data is empty, And is there connection to the type 'Any' of the property.

Update :

class code: class Entity(val id:String) extends Serializable{

var index:Any = null

var geometry:Any = null

}

geometry contains centeroid, shape and coordinates(complex object)

Community
  • 1
  • 1
david lopez
  • 405
  • 2
  • 5
  • 16
  • 2
    If you want us to help, you need to provide a [MCVE] of your problem. Words are great, but we need to actually see what's going on in the code to help. – Yuval Itzchakov May 29 '16 at 09:13

1 Answers1

-1

You should not use Kryo with Scala since the behavior of many Scala classes differs from Java classes and Kryo was originally written to work with Java. You will probably encounter many weird issues like this one if you use Kryo with Scala. You should instead use chill-scala which is an extension of Kryo that handles all of Scala's special cases.

ilooner
  • 2,480
  • 15
  • 32