Non-blocking reading from the file using Reactor

Question

Do you know if it's possible to create Flux of file lines really non-blocking way?

The best solution I've found is next:

Flux.using(
           () -> Files.lines(PATH),
           Flux::fromStream,
           Stream::close
 );

Even though it looks non-blocking but it is blocking under the hood.

https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#lines-java.nio.file.Path-java.nio.charset.Charset-

Read all lines from a file as a Stream. Unlike readAllLines, this method does not read all lines into a List, but instead populates lazily as the stream is consumed.

The returned stream encapsulates a Reader.

Is it possible to create Flux from AsynchronousFileChannel?

Thank you in advance

Why do you think it is blocking? Even this article describes this technique: https://www.baeldung.com/java-nio2-async-file-channel. — Artem Bilan, Feb 01 '22 at 21:58
Hey, Artem. I don't think AsynchronousFileChannel is blocking, I tell about Files.lines(PATH) in a code example above — Ksu, Feb 01 '22 at 22:09
Could you please provide a **minimal complete example** to illustrate the problem? In particular, what schedulers are used for the `publishOn()` and `subscribeOn()` operators? — Sergey Vyacheslavovich Brunov, Feb 02 '22 at 00:56
I don’t think a scheduler should be discussed over here. I want to see first of all any arguments that stream of those lines is blocked. — Artem Bilan, Feb 02 '22 at 03:24
I don't get how Files.lines would be non-blocking. If you use it without Reactor, it will be executed on the main thread, and opening and reading from a file is certainly blocking, there is no callback, future, etc which would indicate otherwise. Java stream doesn't make it non-blocking as it has no async support, does it? — Martin Tarjányi, Feb 02 '22 at 07:05

score 1 · Answer 1 · answered Feb 06 '22 at 18:56

If you have Spring Framework on classpath, then you can do the following:

import org.springframework.core.codec.StringDecoder;
import org.springframework.core.io.buffer.DataBufferUtils;
import org.springframework.core.io.buffer.DefaultDataBufferFactory;

import java.nio.channels.AsynchronousFileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class AsyncFileRead {
    public static void main(String[] args) {
        StringDecoder stringDecoder = StringDecoder.textPlainOnly();

        DataBufferUtils.readAsynchronousFileChannel(() -> AsynchronousFileChannel.open(Path.of("test/sample.txt"),
                StandardOpenOption.READ), DefaultDataBufferFactory.sharedInstance, 4096)
            .transform(dataBufferFlux -> stringDecoder.decode(dataBufferFlux, null, null, null))
            .blockLast();
    }
}

Alternatively, you can use RxIo library which provides a nice abstraction similar to java.nio.file.Files just with async/reactive support:

import org.javaync.io.AsyncFiles;
import reactor.core.publisher.Flux;

import java.nio.file.Path;

public class AsyncFileRead {
    public static void main(String[] args) {
        Flux.from(AsyncFiles.lines(Path.of("test/sample.txt")))
            .blockLast();
    }
}

Although, it's important to note that even these solutions are not truly non-blocking as depending on the platform (Windows, Linux) AsynchronousFileChannel implementation can be blocking under the hood but at least it delegates that task to a dedicated thread pool.

Non-blocking reading from the file using Reactor

1 Answers1