2

With Azure Data Factory I have a CosmosDb Dataset and picked a Collection from the dropdown list.

My Pipeline contains an Activity Copy Data and picked the Source Dataset mentioned above. Option 'Use Query' is set to Query so I have a textbox available.

When I enter a simple query like select c.Name from c I don't get any data and I don't understand why. How can I get the Name property from the dataset so it can be mapped?

This is the dataset: available data

321X
  • 3,153
  • 2
  • 30
  • 42

2 Answers2

0

Based on Supported capabilities in document: Copy data to or from Azure Cosmos DB by using Azure Data Factory, Azure Cosmos DB connector only supports Copying data from and to the Azure Cosmos DB SQL API. That's the reason that you saw the strange results.

So, you could choose mongo connector to do your jobs which is mentioned in this case:https://social.msdn.microsoft.com/Forums/security/en-US/52cddbf7-c132-490c-9088-65a38f9b7200/copy-activity-to-cosmosdb-with-mongo-api?forum=AzureDataFactory.

Jay Gong
  • 23,163
  • 2
  • 27
  • 32
  • 1
    I think I don't understand what you are saying. There is a simple query (what should be supported by SQL API? and is in the docs): `select c.Name from c`... I already use the Mongo connector – 321X Nov 20 '18 at 09:08
  • @321X Your screenshot in the question is for mongo connector?or cosmos db connector? – Jay Gong Nov 20 '18 at 09:25
  • It's a Cosmos database and I use the Cosmos connector... I see I mentioned the Mongo connector in my reply, that's incorrect; my bad! – 321X Nov 20 '18 at 10:14
  • @321X Never mind,so take a shot on mongo connector to see what's happening. – Jay Gong Nov 20 '18 at 10:20
  • I see it's a MongoDb model, I'm going to try the Mongo connector now! – 321X Nov 20 '18 at 10:20
  • Yes, that was the correct hint! I completely understand now what you mean :-) – 321X Nov 20 '18 at 10:39
  • @321X Good Job! – Jay Gong Nov 20 '18 at 12:19
-1

What is your precise requirement ?

  1. Table and Query is single selection, if you select Query, the table(collection) you defined in dataset will not work.

  2. if you select a collection in dataset, and imported schema, you can switch to mapping in the copy activity, if you can't see any mapping, you should make sure you have selected a sink dataset and the sink has schema.

  3. The preview window you opened in dataset will not work for the query in activity source. it's different scenario. so no matter what you input in the query, it will not update

Atvoid
  • 187
  • 1
  • 8
  • 1
    This is a simple use case (select one column) I try to get to work. Next step is that I need to map a collection from document to a File based Dataset. Regarding the Activity: I have the Query option selected and my Dataset doesn't have a schema defined, so all columns should be available to query right. The preview window I opened is the one from the Activity, but I think I understand what you mean. You mean it's related to Dataset and not the Query? When the Query is edited the preview window gets updated though... – 321X Nov 15 '18 at 09:54