Using mongomirror to sync collections within Atlas

Question

I want to migrate a collection from one Mongo Atlas Cluster to another. How do I go about in doing that?

Answer 1

There are 2 poosible approaches here:

1. Migration with downtime: stop the service, export the data from the collection to some 3rd location, and then import the data into the new collection on the new cluster, and resume the service.
2. But there's a better way: using the MongoMirror utility. With this utility, you can sync collections across clusters without any downtime. the utility first syncs the db (or selected collections from it) and then ensures subsequent writes to the source are synced to the dest.

following is the syntax I used to get it to run:

./mongomirror --host atlas-something-shard-0/prod-mysourcedb--shard-00-02-pri.abcd.gcp.mongodb.net:27017 \
   --username myUserName \
   --password PASSWORD \
   --authenticationDatabase admin \
   --destination prod-somethingelse-shard-0/prod-mydestdb-shard-00-02-pri.abcd.gcp.mongodb.net:27017 \
   --destinationUsername myUserName \
   --destinationPassword PASSWORD \
   --includeNamespace dbname.collection1 \
   --includeNamespace dbname.collection2 \
   --ssl \
   --forceDump

Unfortunately, there are MANY pitfalls here:

1. ensure your user has the correct role. this is actually covered in the docs so read the relevant section closely.
2. to correctly specify the host and destination fields, you'll need to obtain both the RS name and the primary instance name. One way to get these is to use the mongosh tool and run rs.conf() on both source and destination clusters. The RS name is specified as "_id" in the command's output, and the instances are listed as "members" in the output. You'll want to take the primary instance's "host' field. the end result should look like RS-name/primary-instance-host:port
3. IF you specify replica-set, you MUST specify the PRIMARY instance. Failing to do so will result in an obscure error (EOF something).
4. I recommend adding the forceDump flag (at least until you manage to get it to run for the first time).
5. If you specify non-existing collections, the utility will only give one indication that they don't exist and then go on to "sync" these, rather than failing.