Flux from CompleteableFuture>

Question

I'm using a Java client that returns a CompleteableFuture (StreamedQueryResult extends Publisher). And I'm trying to convert this to a flux and emit values as they're received. The client connection will be infinite so I believe the issue might be that the CompleteableFuture is never completed? If that's the cause how can I convert this stream of values into a Flux and emit those as they're received?

client.streamQuery() returns CompleteableFuture<? extends Publisher<?>>

Here is my findAll() method where I call the client. It successfully prints received as expected.

    @Override
    public Flux<Person> findAll() {
        return Flux.from(client.streamQuery("select * from deal_change emit changes;").join()).flatMap(row -> {
            System.out.println("received");
            try {
                return Flux.just(mapper.readValue(row.asObject().toJsonString(), Person.class));
            } catch (JsonProcessingException e) {
                return Flux.empty();
            }
        });
    }

Here is the example where I'm calling the method above. But nothing is ever returned and none of the below is ever printed.

        var repo = new PersonRepository(client);
        var person = repo.findAll().map(p -> {
            System.out.println("test");
            return p;
        }).blockFirst();
        System.out.println(person);
        System.out.println("end");

First you can try to get streamQueryResult as Mono, using [Mono.fromFuture](https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#fromFuture-java.util.concurrent.CompletableFuture-). Then, try to block on it. It will allow you to validate the CompletableFuture completes and return expected *StreamedQueryResult*. If it works, you can then [flatMapMany](https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#flatMapMany-java.util.function.Function-)` to transform it into a flux of rows (possibly using Flux.using if you need manual closing). — amanin, Feb 22 '23 at 08:13
Unfortunately that does not work as the completablefuture never completes. I did find a solution though — andresmonc, Feb 22 '23 at 16:24

score 0 · Answer 1 · answered Feb 22 '23 at 16:26

I was able to find a solution by using a Subscriber<?>

    public <T> Flux<T> select(Query query, Class<T> entityClass) throws DataAccessException {
        return Flux.create(sink -> {
            client.streamQuery("select * from blabla emit changes;").thenAccept(streamedQueryResult -> {
                RowSubscriber subscriber = new RowSubscriber(sink);
                streamedQueryResult.subscribe(subscriber);
            });
        }).map(io.confluent.ksql.api.client.Row.class::cast).flatMap(row -> {
            try {
                return Flux.just(objectMapper.readValue(row.asObject().toJsonString(), entityClass));
            } catch (Exception e) {
                return Flux.empty();
            }
        });
    }

Using Flux.create() We use the sink to emit values within the Subscriber of the completablefuture

public class RowSubscriber implements Subscriber<Row> {

    private Subscription subscription;
    private FluxSink sink;

    public RowSubscriber(FluxSink sink) {
        this.sink = sink;
    }

    @Override
    public synchronized void onSubscribe(Subscription subscription) {
        System.out.println("Subscriber is subscribed.");
        this.subscription = subscription;

        // Request the first row
        subscription.request(1);
    }

    @Override
    public synchronized void onNext(Row row) {
        System.out.println("Received a row!");
        System.out.println("Row: " + row.values());
        sink.next(row);
        // Request the next row
        subscription.request(1);
    }

    @Override
    public synchronized void onError(Throwable t) {
        System.out.println("Received an error: " + t);
    }

    @Override
    public synchronized void onComplete() {
        System.out.println("Query has ended.");
    }
}

as onNext() is called the values are emited to the sink.

Flux from CompleteableFuture>

1 Answers1