Spring WebFlux with ReactiveMongoRepository: not getting database updates through the stream

Question

The issue I am facing:

Whatever I try through various tutorials on using Spring Reactive (WebFlux) REST API's, I am unable to get it to work. When I initially call my endpoint, I am able to get the results from the MongoDB collection. However, whenever I make an update to a document entry, or add a new document, it is not updated in through the text-event-stream. Each time I have to call the endpoint again to get new results.

The setup:

Currently I have the following setup:

Spring Cloud Gateway (behind which I run various )
Spring Boot Service containing RestControllers (calling this the main service)

I am using Spring Webflux, Spring Cloud Gateway and Spring ReactiveMongoRepository.

The dependencies included pom.xml for the Spring Boot main service:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-oauth2-jose</artifactId>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb-reactive</artifactId>
        </dependency>

The code for the ReactiveMongoRepository:

@Repository
public interface TestRepository extends ReactiveMongoRepository<TestIntegration, String> {

    @Query(("{'userId': ?0}"))
    Flux<TestIntegration> findbyUserId(String userId);
}

The code for the rest Controller:

@GetMapping(value = "main/integrations", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<TestIntegration> retrieveIntegrations(ServerWebExchange exchange) {
        return testRepository.findAll();
}

According to every tutorial/guideline I have read, this should be working according to this approach. Has anyone experienced this as well or might be able to assist in the matter? Currently stuck on this for days...

That works exactly as intended, you open a stream to you application and ask for database entries. But how would your application know there has been new entries written to the database? Well it doesnt. You need to like all applications trigger event when there are new enteies and place them in the stream. — Toerktumlare, Jan 22 '21 at 07:32
That's interesting and good to know that it's working as intended. Thanks for the clarification. Do you have perhaps some information on how those trigger events can be implemented, some documentation and/or pointers perhaps? So that I at least know in which direction to look. — vv01, Jan 22 '21 at 09:40
Stack overflow is not a coding forum, it is a Q&A site, so start by googling reading up what works for your database. I have no idea how it works for your db. You are also not allowed to ask for books, tutorials or documentation, its not that kind of site. read up on the rules of stack overflow. If you want to ”discuss” problems then you have to find another site. — Toerktumlare, Jan 22 '21 at 10:24
I didn't ask for that specifically, but nevermind. Just wondering which way is the right way to search for in order to make this work. So far I have run into changeStreams for ReactiveMongoTemplate. Will figure it out. — vv01, Jan 22 '21 at 14:52
`Just wondering which way is the right way to search for in order to make this work` is asking for documentation, tutorials and/or libraries which is not allowed. SO is a coding forum for very specific questions, with one specific answer. Your question has been answered. For future reference https://meta.stackoverflow.com/questions/261592/how-much-research-effort-is-expected-of-stack-overflow-users — Toerktumlare, Jan 22 '21 at 15:00

Toerktumlare · Accepted Answer · 2021-01-23T17:04:21.400

This works as intended. When you stream items from a Flux<T> it will feed as long as there are items in the stream. Then the stream will close. So in your case, it fetched all data from the database, streamed it to you and then closed.

If you wish to keep the stream open you need to keep sending data. One way to do that is to send :keep alive messages (messages that start with a comma) using ServerSentsEvents. You can read more about ServerSentEvents and the colon operator and these types of messages in the official mozilla documentation.

When you are able to actually keep the stream open, and wish to send data, your service will not know when new data is written to the database. So either you poll your database, or trigger an event when something is written, to fetch the newly written data and place this in the stream.

How to place data in a continuous open stream is way too big of a topic to explain here. But i suggest you read the following sections in the official reactor documentation:

Programmatically creating a sequence

Processors and Sinks

That's really helpful. Thanks a lot in helping to point in the right direction. Much appreciated. — vv01, Jan 22 '21 at 23:34

Spring WebFlux with ReactiveMongoRepository: not getting database updates through the stream

1 Answers1