Serialize multiple objects into a single file using Kryo

Question

As far I know, Kryo serialization / deserialization happens per object. Is it possible to serialize multiple objects into a single file?. One of workaround suggested in another similar SO question was to use an array of objects. Considering a huge amount of data that needs to be serialized, I feel it would not be as efficient as it should be. Is it right assumption?

score 2 · Answer 1 · answered Jan 07 '14 at 09:01

As Kryo supports streaming there is nothing to stop you writing/reading more than one object to kryo "at the top level". For example the following program writes two unrelated objects to a file and then deserializes them again

public class TestClass{


    public static void main(String[] args) throws FileNotFoundException{
        serialize();
        deSerialize();
    }

    public static void serialize() throws FileNotFoundException{
        Collection<String>collection=new ArrayList<>();
        int otherData=12;


        collection.add("This is a serialized collection of strings");

        Kryo kryo = new Kryo();
        Output output = new Output(new FileOutputStream("testfile"));
        kryo.writeClassAndObject(output, collection);
        kryo.writeClassAndObject(output, otherData); //we could add as many of these as we like
        output.close();
    }

    public static void deSerialize() throws FileNotFoundException{
        Collection<String>collection;
        int otherData;

        Kryo kryo = new Kryo();
        Input input = new Input(new FileInputStream("testfile"));
        collection=(Collection<String>)kryo.readClassAndObject(input);
        otherData=(Integer)kryo.readClassAndObject(input);

        input.close();

        for(String string: collection){
            System.out.println(string);
        }

        System.out.println("There are other things too! like; " + otherData);

    }


}

score 2 · Answer 2 · answered Mar 05 '11 at 04:26

2

Does Kryo API take an OutputStream? If so, just feed it the same OutputStream to serialize multiple files. Do the same with InputStream when reading. A good serialization format will have length encodings or termination symbols and would not rely on EOF for anything.

The array approach would also work with minimal overhead as long as all of these objects are already in memory. You are talking about adding just a few bytes per object to create an array to hold them. If they aren't all in memory, you would have to load them all into memory first to create an array around them. That could definitely become a problem given large enough data set.

answered Mar 05 '11 at 04:26

Konstantin Komissarchik

28,879
6
61
61

The buffer / stream during serialization / deserialization is tightly mapped with class. The above approach may not work. Let me try. – Harsha Hulageri Mar 09 '11 at 00:34
Objects are not in memory. So array approach will be a problem – Harsha Hulageri Mar 09 '11 at 00:35
3

This answer is correct, Kryo v2 supports streaming. Write the objects sequentially to a file. Read them one by one from the file. – NateS Jun 15 '12 at 02:47

Serialize multiple objects into a single file using Kryo

2 Answers2