How do I detect "End of File" in Kryo?

Question

I'm reading an unknown number of objects from a gzipped file in Kryo:

Input input = new Input(new GZIPInputStream(Files.newInputStream(inputFile)));

The problem is: How do I detect when I've read the last object from the file? When I'm writing the objects I don't know in advance how many I will be writing, so I can't include a count of the objects at the start of the file.

I could have some kind of "dummy" object that I write at the end of the file to indicate that it's the last one, but that seems ugly.

What does it say in the Javadoc? – user207421 Nov 09 '12 at 21:01 — user207421, Nov 09 '12 at 21:01

score 3 · Answer 1 · answered Jun 19 '13 at 21:58

3

Use Input#eof() after reading each object to check if you hit the end of the stream.

answered Jun 19 '13 at 21:58

NateS

5,751
4
49
59

Muffintop · Answer 2 · 2013-11-18T20:23:18.053

1

For some reason, input.eof() does not effectively detect EOF, for example, this code resulted in com.esotericsoftware.kryo.KryoException: Buffer underflow.

while (!input.eof())
{
    SomeClass someObject = kryo.readObject(input, SomeClass.class);
    // ...
}

As a workaround, I did something like this:

while (true)
{
    try
    {
        SomeClass someObject = kryo.readObject(input, SomeClass.class);
        // ...
    }
    catch(KryoException ke)
    {
        break;
    }
}

EDIT: I was using the jar version 2.21, but eof worked with 2.22.cloudera.1.

edited Nov 18 '13 at 20:23

answered Nov 18 '13 at 20:17

Muffintop

543
4
11

Did you see this `Buffer underflow` ever since? I'm using 2.22 version but it seems the exception is still here. It appears from time to time. The code is not run from multiple threads. @NateS, any hints? – dcernahoschi Mar 12 '14 at 12:31

Atul Soman · Answer 3 · 2015-10-16T06:33:00.657

1

For latest (3.0.3) kryo libraries you can use input.eof(). For older ones use input.available() with a cost of performance (Almost 8 times slower).

while(input.available() > 0){
  someObject = kryo.readObject(input, SomeClass.class);
  i++;
}

edited Oct 16 '15 at 06:33

answered Oct 16 '15 at 06:22

Atul Soman

4,612
4
30
45

How do I detect "End of File" in Kryo?

3 Answers3