MongoDB Change Stream reading data which are old updates

Question

I have created an application which reads mongo change stream for updates and inserts and then we take action on the changed data. Below is my code snippet

private void listenChangeStream() {
        Runnable changeListener = new Runnable() {
            @Override
            public void run() {
                String fullDoc = null;
                String updateInfo = null;

                while (cursor.hasNext()) {
                    try {
                        ChangeStreamDocument<Document> next = cursor.next();
                        String id = next.getDocumentKey().getString("id").getValue();
                        LOGGER.debug("Change Stream recived:{}", next);
                        String operationType = next.getOperationType().getValue();
                        if ("insert".equals(operationType) || "replace".equals(operationType)) {
                               fullDoc = next.getFullDocument().toString();
                            if (fullDoc.contains("image_info")) {
                                kafkaProducer
                                        .pushOfflineProcessingData(new DataPackets(Id, OfflineProcessType.IMAGE));
                            }
                        } else if ("update".equals(operationType)) {
                               updateInfo = next.getUpdateDescription().toString();
                            if (updateInfo.contains("image_info"))
                                kafkaProducer
                                        .pushOfflineProcessingData(new DataPackets(Id, OfflineProcessType.IMAGE));
                        } 

                    } catch (Exception ex) {
                        LOGGER.info("Exception has come in cahnge listener::", ex);
                    }
                }

            }
        };
        executor = Executors.newFixedThreadPool(1);
        executor.execute(changeListener);

    }

private MongoCursor<ChangeStreamDocument<Document>> getCursor(MongoCollection<Document> supplierCollection, List<Bson> pipeline) {
        MongoCursor<ChangeStreamDocument<Document>> cursor;     
             cursor = supplierCollection.watch(pipeline).iterator();        
        return cursor;
    }

This is working fine. The problem which i am facing is when ever i start the server the change stream starts reading old committed changes . Which i do not want. I want after the deployment only the new updates should be picked by this.

Can any one suggest how to do it?

score 3 · Answer 1 · answered Jul 12 '18 at 05:02

Can any one suggest how to do it?

In MongoDB v4.0 with MongoDB Java driver v3.8, you can specify startAtOperationTime parameter to MongoClient.watch().

The change stream will only provides changes that occurred after the specified timestamp. Any command run against the MongoDB server will return an operation time that can be used as a value of the parameter. The default value is an operation time obtained from the server before the change stream was created.

Alternatively, you could also cache the last seen _id from the change stream notification. This is a resumeToken that you could pass to resumeAfter() method to resume notifications starting after the operation specified in the resumeToken. For example:

BsonDocument resumeToken = next.getResumeToken();
cursor = inventory.watch().resumeAfter(resumeToken).iterator();

MongoDB Change Stream reading data which are old updates

1 Answers1