Read file line by line using non-blocking IO with reactive wrapper in Java

Question

Is there any way to read a local file line by line without blocking any thread (background thread pool counts as blocking) using built-in CompletableFuture or a reactive stream library like RxJava or Reactor?

(It's interesting that there are many non-blocking IO libraries for HTTP and different databases like Mongo, Redis, etc., but I wasn't able to find anything for a simple file read.)

You can't. There is no non-blocking I/O on files in Java. There is however *asynchronous* I/O, via `AsynchronousFileChannel`, but it doesn't support reading lines. Why do you think you need this? — user207421, Apr 13 '19 at 20:48

Alexander Sorkin · Accepted Answer · 2019-04-15T11:12:20.647

2

There are similar questions:

The main reason why Java has no universal non-blocking file IO is the following: Java is a cross-platform language, but Unix does not have non-blocking access to files.

If you program for Windows, there is a platform-specific implementation WindowsAsynchronousFileChannelImpl, that uses a non-blocking mechanism.

edited Apr 15 '19 at 11:12

answered Apr 15 '19 at 10:47

Alexander Sorkin

634
7
20

Not really sure where you came up with the "Linux/Unix does not have non-blocking file IO" part, You can use `AsynchronousFileChannel` on Linux as well and it has been there at least since JDK 8. As far as I know you should be good to go if you have a kernel 2.6+. Do you have any articles I can check regarding the non-blocking file IO you mentioned? :) – tftd Jun 11 '22 at 12:45

score 2 · Answer 2 · answered Feb 10 '22 at 18:33

Although, I got answer for my question from Alexander, I'd like to share the most reactive ways that I found to read a file line by line. Both solutions use AsynchronousFileChannel under the hood which is not always non-blocking but still okay to use in a reactive environment since it uses a dedicated thread pool for IO work.

Using utils from Spring Framework (ideal in WebFlux application)

import org.springframework.core.codec.StringDecoder;
import org.springframework.core.io.buffer.DataBufferUtils;
import org.springframework.core.io.buffer.DefaultDataBufferFactory;

import java.nio.channels.AsynchronousFileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class AsyncFileRead {
    public Flux<String> lines() {
        StringDecoder stringDecoder = StringDecoder.textPlainOnly();

        return DataBufferUtils.readAsynchronousFileChannel(() -> AsynchronousFileChannel.open(Path.of("test/sample.txt"),
                StandardOpenOption.READ), DefaultDataBufferFactory.sharedInstance, 4096)
            .transform(dataBufferFlux -> stringDecoder.decode(dataBufferFlux, null, null, null));
    }
}

Using RxIo library

import org.javaync.io.AsyncFiles;
import reactor.core.publisher.Flux;

import java.nio.file.Path;

public class AsyncFileRead {
    public Flux<String> lines() {
        return Flux.from(AsyncFiles.lines(Path.of("test/sample.txt")));
    }
}

Read file line by line using non-blocking IO with reactive wrapper in Java

2 Answers2

Using utils from Spring Framework (ideal in WebFlux application)

Using RxIo library

Linked