Stack Overflow
Stack Overflow

Questions tagged [kryo]

Kryo is a fast and efficient object graph serialization framework for Java. The goals of the project are speed, efficiency, and an easy to use API. The project is useful any time objects need to be persisted, whether to a file, database, or over the network.

431 questions
4
votes
1 answer

How can I register classes to Kryo Serializer in Apache Spark?

I am using Spark 1.6.1 and Python. How can I enable Kryo serialization when working with PySpark? I have the following settings in the spark-default.conf file: spark.eventLog.enabled true spark.eventLog.dir …
User2130
  • 555
  • 1
  • 6
  • 16
4
votes
0 answers

com.esotericsoftware.kryo.KryoException: Unable to find class: ����(

I am currently encountering the following exception: eglSurfaceAttrib not implemented Failed to set EGL_SWAP_BEHAVIOR on surface 0x9e878060, error=EGL_SUCCESS com.esotericsoftware.kryo.KryoException: Unable to find class: ����( Serialization…
Vic Torious
  • 1,757
  • 3
  • 21
  • 24
4
votes
2 answers

How do I use joda.time in flink (or how do I use typeutils.runtime.kryo)

In a flink project I use a case class click. case class click( date: LocalDateTime, stbId:String, channelId :Int) This class populated DataSets and it worked fine with the date being a java 8 java.time.LocalDateTime. After switching to org.joda…
Spyros Komninos
  • 137
  • 9
4
votes
1 answer

Kryo serialization/deserialization

I'm trying to use kryo to serialize and deserialize to binary. I think I have the serialization working but I cannot seem to deserialize. Below is the code I'm messing around with but eventually I want to store a byte[] and later read it again. The…
Madmenyo
  • 8,389
  • 7
  • 52
  • 99
4
votes
0 answers

Kryo deserializer throws ClassNotFoundException in spark: Unable to find class

I am writing a spark job in scala to be run with spark 1.3.0. My RDD transformation functions use classes from a third party library which are not Serializable. To make closure serialization possible, I wrap these objects in…
saurabh
  • 83
  • 1
  • 7
4
votes
0 answers

Map of Avro objects serialization with Kryo - Spark

I have a question about specific avro serialization with kryo. Sorry if this question seems redundant but I didn't find a simple answer. I've configured Spark to run with Kryo and it works well : Avro generated classes are well serialized using a…
Seb
  • 378
  • 5
  • 13
4
votes
2 answers

Java, Kryo: Add fields to class persisted as FieldSerializer

This is using the Kryo serialization library. I've got objects already serialized to the DB with FieldSerializer. Now, I'd like to add a couple fields to the serialized classs, and obtain backward compatibility with them. Is this possible? What…
mtyson
  • 8,196
  • 16
  • 66
  • 106
4
votes
2 answers

Serializing an arbitrary Java object with Kryo (getting IllegalAccessError)

Motivation: To aid in remote debugging (Java), it's useful to be able to request remote servers to send over arbitrary objects to my local machine for inspection. However, this means that the remote server must be able to serialize an arbitrary java…
mchen
  • 9,808
  • 17
  • 72
  • 125
4
votes
0 answers

How do you deserialize Kryo into case classes using Scalding?

I know that Scalding's default serialization uses Kryo. So for this example, lets say I have a pipe of student objects. case class Student(name:String, id:String) val pipe: Pipe[Student] = //.... Then I write that pipe to a TextDelimited file…
user3335040
  • 649
  • 1
  • 7
  • 17
4
votes
1 answer

sparkContext broadcast JedisPool not work

I was use sparkContext.broadcast in my spark streaming application for share redis connection pool(JedisPool). The code like this: lazy val redisPool = { val pool = Redis.createRedisPool(redisHost, redisPort) …
Guy
  • 53
  • 1
  • 6
4
votes
0 answers

Using Kryo to serialize a Spark mllib model

I want to serialize a model which is generate from ML lib Classifacation Tree, A model is construct by a Node and a Algo, Node seems serializable Can we just serialize a whole model to disk and read from another class to get the model back? Thanks
Wilson Liao
  • 608
  • 1
  • 6
  • 25
4
votes
1 answer

Kryo vs Jackson

I am checking Java Serialization, Kryo and Jackson. I have created some tiny code which serialize random objects N times. I measure the time to serialize and deserialize and the size of the object. I'm not pretty happy about the size Kryo vs…
Guille
  • 2,248
  • 3
  • 24
  • 42
4
votes
1 answer

Sending and receiving arrays in kryonet

I'm using kryonet to create a Server/Client system. I'm trying to send an int[][] to my client, from the server. To achieve this task I tried this: publc class ArrayPacket { public int[][] array } Then, in the server listener I wrote: ArrayPacket…
A Cat
  • 629
  • 7
  • 20
4
votes
1 answer

kryo serializing of class (task object) in apache spark returns null while de-serialization

I am using java spark API to write some test application . I am using a class which doesn't extends serializable interface . So to make the application work I am using kryo serializer to serialize the class . But the problem which I observed while…
Harsh Gupta
  • 339
  • 4
  • 20
4
votes
0 answers

Java serialization of very large objects

I'm facing the following issue: I would like to serialize a very large object (several hundreds of MB in memory) to a file. As I understood that Kryo is one of the best serialization libraries out there, I've been using this to serialize my…
Tom
  • 1,375
  • 3
  • 24
  • 45
1 2 3
…
28 29