Stack Overflow
Stack Overflow

Questions tagged [kryo]

Kryo is a fast and efficient object graph serialization framework for Java. The goals of the project are speed, efficiency, and an easy to use API. The project is useful any time objects need to be persisted, whether to a file, database, or over the network.

431 questions
3
votes
1 answer

Spark kryo_serializers and Broadcast>> java.io.IOException: java.lang.UnsupportedOperationException

When I try to access a Broadcast variable I receive this Exception: 17/03/26 03:04:23 WARN TaskSetManager: Lost task 0.0 in stage 3.0 (TID 10, 192.168.56.5, executor 1): java.io.IOException: java.lang.UnsupportedOperationException at…
Piero Zito
  • 67
  • 1
  • 6
3
votes
0 answers

KryoSerializer cannot find my SparkKryoRegistrator

I am using Spark 2.0.2 on Amazon emr-5.2.1 in client mode. I use Kryo serialisation and register our classes in our own KryoRegistrator: val sparkConf = new SparkConf() .set("spark.serializer", "org.apache.spark.serializer.KryoSerializer") …
Holger Jaekel
  • 106
  • 4
3
votes
2 answers

Spark streaming : com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Class is not registered: scala.Tuple2$mcJZ$sp

I am trying to use Kryo Serializer in spark streaming. I read in Spark tuning docs that - Finally, if you don’t register your custom classes, Kryo will still work, but it will have to store the full class name with each object, which is…
Nishant Kumar
  • 2,199
  • 2
  • 22
  • 43
3
votes
1 answer

Kryo throwing " Encountered unregistered class ID: 44"

Kyro is throwing me the following message: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 44 at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137) at…
raycons
  • 735
  • 12
  • 26
3
votes
1 answer

Java 8 Base64 stream wrapper eats last two chars?

I have the following code: public static String encode(Object object) { final String result; try (ByteArrayOutputStream bOut = new ByteArrayOutputStream(); Output output = new Output(bOut)) { KRYO.writeObject(output,…
Pinkie Swirl
  • 2,375
  • 1
  • 20
  • 25
3
votes
3 answers

Serialization of an object used in foreachRDD() when CheckPointing

According to this question and documentations I've read, Spark Streaming's foreachRDD(someFunction) will have someFunction itself executed in the driver process ONLY, though if there were operations done on RDDs then they will be done on the…
howard
  • 255
  • 1
  • 4
  • 12
3
votes
1 answer

Deserializing Java objects from Kafka consumer

I have a Kafka Consumer, currently configured with: kafkaProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); kafkaProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,…
Matt
  • 3,303
  • 5
  • 31
  • 53
3
votes
2 answers

Error using Spark's Kryo serializer with java protocol buffers that have arrays of strings

I am hitting a bug when using java protocol buffer classes as the object model for RDDs in Spark jobs, For my application, my ,proto file has properties that are repeated string. For example message OntologyHumanName { repeated string family =…
eboni
  • 883
  • 2
  • 10
  • 25
3
votes
1 answer

How to send Kryo serialized objects over JMS?

I would like to use Kryo to (de-)serialize objects and send/receive them via JMS. The problem I'm having is that both sides, sender and receiver, must register the classes with the same ID. Kryo has a method register (Class type, int id) that I use.…
Frizz
  • 2,524
  • 6
  • 31
  • 45
3
votes
1 answer

Spark Kryo Serialization fails

I have a piece of Spark code that worked on Spark 1.3 but fails when I move it to Spark 1.5.2 (cluster upgrade out of my control). The failure is as follows: Caused by: java.io.NotSerializableException: com.location.model.Profile Serialization…
Daniel Langdon
  • 5,899
  • 4
  • 28
  • 48
3
votes
1 answer

Kryonet. RMI. Response timed out while trying to execute remote method

I`m trying to bootstrap simple kryonet RMI project with two separate client and server applications. But when I try to execute remove method - it leads to: Exception in thread "main" com.esotericsoftware.kryonet.rmi.TimeoutException: Response timed…
Viktor Sidochenko
  • 361
  • 3
  • 17
3
votes
0 answers

kryo Encountered unregistered class ID, even though I've registered all classes

I have a spark job that serializes an object with setRegistrationRequired(true). I have added registrations for all the classes. I have java code that deserializes the serialized object, also with setRegistrationRequired(true) and all appropriate…
Martin Klosi
  • 3,098
  • 4
  • 32
  • 39
3
votes
0 answers

How do I intercept deserialization in Kryo based on the super class?

I am using the following code to add hooks to Kryo deserialization. private static class KryoWithHooks extends Kryo { private static final class SerializerWithHook extends com.esotericsoftware.kryo.Serializer { private final…
Sergey Shelukhin
  • 31
  • 2
3
votes
1 answer

Kryonet - buffer overflow - small object sending

Hiho. Everything was running without problems on localhost. Now the server is set on a professional host. The error is always the same: 10-04 16:35:29.974 5730-5830/com.drkmns.gameloopballs E/AndroidRuntime: FATAL EXCEPTION: Thread-8160 10-04…
user5064340
3
votes
1 answer

Exception in Chill-kryo

I am trying to run the sample code for Chill-Scala from this link I have two dummy case classes one is User @SerialVersionUID(1) case class User(name :String, age : Int) extends Serializable and the other is Student @SerialVersionUID(2) case…
M.Ahsen Taqi
  • 965
  • 11
  • 35
1 2 3
…
28 29