How to use AsynchronousFileChannel to read to a StringBuffer efficiently

Question

So you know you can use AsynchronousFileChannel to read an entire file to a String:

 AsynchronousFileChannel fileChannel = AsynchronousFileChannel.open(filePath, StandardOpenOption.READ);
            long len = fileChannel.size();

            ReadAttachment readAttachment = new ReadAttachment();
            readAttachment.byteBuffer = ByteBuffer.allocate((int) len);
            readAttachment.asynchronousChannel = fileChannel;

            CompletionHandler<Integer, ReadAttachment> completionHandler = new CompletionHandler<Integer, ReadAttachment>() {

                @Override
                public void completed(Integer result, ReadAttachment attachment) {

                    String content = new String(attachment.byteBuffer.array());
                    try {
                        attachment.asynchronousChannel.close();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                    completeCallback.accept(content);
                }

                @Override
                public void failed(Throwable exc, ReadAttachment attachment) {
                    exc.printStackTrace();
                    exceptionError(errorCallback, completeCallback, String.format("error while reading file [%s]: %s", path, exc.getMessage()));
                }
            };

            fileChannel.read(
                    readAttachment.byteBuffer,
                    0,
                    readAttachment,
                    completionHandler);

Suppose that now, I don't want to allocate an entire ByteBuffer, but read line by line. I could use a ByteBuffer of fixed width and keep recalling read many times, always copying and appending to a StringBuffer until I don't get to a new line... My only concern is: because the encoding of the file that I am reading could be multi byte per character (UTF something), it may happen that the read bytes end with an uncomplete character. How can I make sure that I'm converting the right bytes into strings and not messing up the encoding?

UPDATE: answer is in the comment of the selected answer, but it basically points to CharsetDecoder.

Don't use async I/O to read lines. It is just not suitable. You can read millions of lines per second with `BufferedReader.readLine().` — user207421, Oct 18 '15 at 23:24
Then why are you using async I/O? That's not non-blocking. It is a third paradigm, after blocking and non-blocking. But why do you think you can't use blocking I/O in the first place? — user207421, Oct 19 '15 at 09:53
If I use the Handler it is supposed to be non blocking no? What other paradigm do you suggest? — gotch4, Oct 19 '15 at 14:34

Zbynek Vyskovsky - kvr000 · Accepted Answer · 2015-10-20T06:23:22.220

1

If you have clear ASCII separator which you have in your case (\n), you'll not need to care about incomplete string as this character maps to singlebyte (and vice versa).

So just search for '\n' byte in your input and read and convert anything before into String. Loop until no more new lines are found. Then compact the buffer and reuse it for next read. If you don't find new line you'll have to allocate bigger buffer, copy the content of the old one and only then call the read again.

EDIT: As mentioned in the comment, you can pass the ByteBuffer to CharsetDecoder on the fly and translate it into CharBuffer (then append to StringBuilder or whatever is preffered solution).

edited Oct 20 '15 at 06:23

answered Oct 19 '15 at 15:20

Zbynek Vyskovsky - kvr000

18,186
3
35
43

In this way anyway I've to store an entire line as a byte buffer... Let's for a moment forget I'm dealing with lines... And that my buffer is limited (lines can be very long). How would we do? – gotch4 Oct 19 '15 at 15:40
1

You can use http://docs.oracle.com/javase/7/docs/api/java/nio/charset/CharsetDecoder.html#decode(java.nio.ByteBuffer,%20java.nio.CharBuffer,%20boolean) to convert the input on the fly. You'll still have to manage the buffer as it might contain remaining characters between the reads. – Zbynek Vyskovsky - kvr000 Oct 19 '15 at 16:12
Perfect! Thanks, consider updating the answer – gotch4 Oct 19 '15 at 21:36

score -2 · Answer 2 · answered Oct 19 '15 at 15:43

-2

Try Scanner:

    Scanner sc = new Scanner(FileChannel.open(filePath, StandardOpenOption.READ));
   String line = sc.readLine();

FileChannel is InterruptibleChannel

answered Oct 19 '15 at 15:43

Evgeniy Dorofeev

133,369
30
199
275

again, I don't need interruptibility, I need to start the read and a callback later... – gotch4 Oct 19 '15 at 21:33

How to use AsynchronousFileChannel to read to a StringBuffer efficiently

2 Answers2