MongoDB Reactive Streams hangs when performing a query

Question

I'm using the MongoDB Reactive Streams Java API which I implemented following this example, but I'm encountering a serious problem: sometimes, when I try to query a collection, the await methods doesn't work, and it hangs until the timeout is reached.

The onSubscribe methods gets called correctly, but then neither onNext, nor onError nor onComplete get called.

There doesn't seem to be a specific circumstance causing this issue.

This is my code

MongoDatabase database = MongoDBConnector.getClient().getDatabase("myDb");
MongoCollection<Document> collection = database.getCollection("myCollection");
FindPublisher<Document> finder = collection.find(Filters.exists("myField"));    
SettingSubscriber tagSub = new SettingSubscriber(finder); 
//SettingsSubscriber is a subclass of ObservableSubscriber which calls publisher.subscribe(this)
tagSub.await(); //this is where it hangs
return tagSub.getWrappedData();

`publisher#subscribe(subscriber)`, is missing. See the [Quick Tour - MongoDB Reactive Streams Java Driver](http://mongodb.github.io/mongo-java-driver-reactivestreams/1.13/getting-started/quick-tour/). Also, see [org.reactivestreams.Publisher's subscribe()](http://www.reactive-streams.org/reactive-streams-1.0.3-javadoc/org/reactivestreams/Publisher.html). — prasad_, Oct 08 '21 at 06:51
@prasad_ Actually it is there (see the comment on the 5th line of code). Also, as I said, `onSubscribe` always gets called. — gscaparrotti, Oct 08 '21 at 07:45
@gscaparrotti How many documents are present in the collection `myCollection`? The exists check on `myField` will perform a full collection scan. If there are a lot of records in the collection, the query is probably timing out — devatherock, Oct 10 '21 at 01:27
There are less than 100 documents, and the timeout is one minute, so I don't think this is the case. — gscaparrotti, Oct 10 '21 at 15:28

score 1 · Answer 1 · answered Oct 14 '21 at 23:42

I wrote a simple implementation of what I assumed the SettingSubscriber looked like and tried to recreate the problem using a groovy script. I couldn't - my code runs without hanging, prints each output record and exits. Code for reference below:

@Grab(group = 'org.mongodb', module = 'mongodb-driver-reactivestreams', version = '4.3.3')
@Grab(group = 'org.slf4j', module = 'slf4j-api', version = '1.7.32')
@Grab(group = 'ch.qos.logback', module = 'logback-classic', version = '1.2.6')

import com.mongodb.MongoClientSettings;
import com.mongodb.MongoCredential;
import com.mongodb.ServerAddress;
import com.mongodb.reactivestreams.client.MongoClients;
import com.mongodb.reactivestreams.client.MongoClient;
import com.mongodb.reactivestreams.client.MongoDatabase;
import com.mongodb.reactivestreams.client.MongoCollection;
import com.mongodb.reactivestreams.client.FindPublisher;
import com.mongodb.client.model.Filters;
import org.bson.Document;
import org.reactivestreams.Subscriber;
import org.reactivestreams.Subscription;

import java.util.concurrent.TimeUnit;
import java.util.concurrent.CountDownLatch;

MongoClientSettings.Builder clientSettingsBuilder = MongoClientSettings.builder()
        .applyToClusterSettings { clusterSettingsBuilder ->
            clusterSettingsBuilder.hosts( Arrays.asList(new ServerAddress("localhost", 27017)))
        };
MongoClient mongoClient = MongoClients.create(clientSettingsBuilder.build());
MongoDatabase database = mongoClient.getDatabase("myDb");
MongoCollection<Document> collection = database.getCollection("myCollection");
FindPublisher<Document> finder = collection.find(Filters.exists("myField"));
SettingSubscriber tagSub = new SettingSubscriber(finder);
tagSub.await();

class SettingSubscriber implements Subscriber<Document> {
    private final CountDownLatch latch = new CountDownLatch(1);
    private Subscription subscription;
    private List<Document> data = new ArrayList<>();

    public SettingSubscriber(FindPublisher<Document> finder) {
        finder.subscribe(this);
    }

    @Override
    public void onSubscribe(final Subscription subscription) {
        this.subscription = subscription;
        subscription.request(1);
    }

    @Override
    public void onNext(final Document document) {
        System.out.println("Received: " + document);
        data.add(document);
        subscription.request(1);
    }

    @Override
    public void onError(final Throwable throwable) {
        throwable.printStackTrace();
        latch.countDown();
    }

    @Override
    public void onComplete() {
        System.out.println("Completed");
        latch.countDown();
    }
    
    public List<Document> getWrappedData() {
        return data;
    }

    public void await() throws Throwable {
        await(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
    }

    public void await(final long timeout, final TimeUnit unit) throws Throwable {
        if (!latch.await(timeout, unit)) {
            System.out.println("Publish timed out");
        }
    }
}

Can you compare this implementation of the SettingSubscriber with yours to see if something is missed?

The only difference I see is that I don't call `subscription.request(1)` in `onNext` — gscaparrotti, Nov 10 '21 at 14:36
Are you doing something like `subscription.request(Long.MAX_VALUE)` once in the `onSubscribe` method instead? The `subscription.request` call needs to be made somewhere, either for all results in one shot like `subscription.request(Long.MAX_VALUE)` or for fewer results at a time like `subscription.request(1)` — devatherock, Nov 10 '21 at 14:52
I do it in the `onSubscribe` method. What happens if I request only a small amount of items once and the publisher wants to send more? — gscaparrotti, Nov 10 '21 at 14:54
That's what the `subscription.request` call in `onNext` method is for, to request more items after we receive an item — devatherock, Nov 10 '21 at 15:48
I get this. I wonder what happens if the Publisher has more available items than the requested amount. — gscaparrotti, Nov 10 '21 at 16:34
I think the subscriber wouldn't receive the additional items unless it requests for them — devatherock, Nov 10 '21 at 19:22

MongoDB Reactive Streams hangs when performing a query

1 Answers1