0

How can i replicate (incremental load) MongoDB (NoSQL) to SQL tables. We have a web-based solution that loading data into MongoDB. The data size is almost 1TB. We need to do BI Reporting in the Looker BI tool. but looker doesn't support MongoDB directly. So we have to replicate our data into SQL form we have redshift for the target database.

Main requirements for parsing NoSQL to SQL:

  1. Parent Node should be the main table
  2. Nested node/arrays should be a separate table with parent key (foreign key)
  3. Whenever a new column is introduced in MongoDB source it should automatically start replicating that new field from any document to the target database.
  4. Incremental refresh from source to target.

I've seen Stitch Data ETL which fits my requirement but I'm looking for OpenSource any ETL/DB tool or library.

Please help.

Adeel Nazir
  • 54
  • 10

1 Answers1

0

Posting answers to help out others with the same requirements.

I'm not able to get any open source ETL tool who can full fill the above 4 requirements. Trying to writing python code to do so. But a paid tool named Precog helped me to fulfill all the above requirements, and a little bit cheaper than Stitch Data ETL. Thanks

Adeel Nazir
  • 54
  • 10