1

I am using monstech version 6 by running a docker container.

It syncs data to elastic search as per my requirement but when i re-install it ( docker stop, docker rm, and then docker run) , it re-syncs the entire data again. I am using direct-read-namespaces mechanism for indexing my mongo collection into elastic search.

I have searched for the suggested mechanism for delta diff (on monstech restart) it but the soln suggested doesn't seem to be working for me.

https://github.com/rwynn/monstache/issues/152

monstache : version: 6 Elastic search : 7.4.2 Mongodb version: MongoDB 4.2

Ikar Pohorský
  • 4,617
  • 6
  • 39
  • 56
Amit Meena
  • 2,884
  • 2
  • 21
  • 33

1 Answers1

1

So what worked for me is changing the configuration from config direct-read-namespaces to change-stream-namespaces.

Instead of tailing oplogs (configured via config direct-read-namespaces ), this mechanism uses the concept of watch api and can be used for very well sync the delta.

Official docs says :

"This option requires MongoDB 3.6 or above. The namespaces included will be tailed using watch API. When this option is enabled the direct tailing of the oplog is disabled, therefore you do not need to specify additional regular expressions to filter the set of collections to watch."

Refer: https://rwynn.github.io/monstache-site/config/

Amit Meena
  • 2,884
  • 2
  • 21
  • 33