2

Problem and Research:

Trying to get real time data from CosmosDB to Data Lake. this is what i have understood from my research, that i have to create a function app to monitor the changes in Cosmos using Change Feed then i have to bind it to event grid, by which i will be able to store the changes to ADLS.

Blockers:

  • In Data Factory, Data flow is not able to connect Cosmosdb Mongo.
  • I have to listen to all the collections, in function app at a time only one collection can be monitored
  • which compute should i use to store the data.

My Understanding: Azure CosmosDb to ADLS pipeline

Have to create a Streaming Pipeline which store all the data from CosmosDB mongoAPI to ADLS storage

David Makogon
  • 69,407
  • 21
  • 141
  • 189
Pushkar Ambatkar
  • 49
  • 5
  • 1
    So... your question is off-topic here, as really it's about finding documentation for doing all the things you want to do. My advice: look at the docs for *Change Streams* (Changefeed is for the SQL API). And also... you don't need to connect Data Factory to your MongoDB API instance, since you're already going to consume the change stream somewhere else. – David Makogon Feb 02 '23 at 13:24

1 Answers1

-1

You can try Azure synapse link for cosmosDB - https://learn.microsoft.com/en-us/azure/cosmos-db/synapse-link

Chen Hirsh
  • 736
  • 1
  • 1
  • 13
  • Please don't post link-only answers (they aren't valid answers, as they really don't *answer* anything). The question is off-topic for several reasons, as well (service recommendation, for one). – David Makogon Feb 03 '23 at 14:49