3

In the Azure environment, I have an Azure SQL Db and a CosmosDb Graph. Using an Azure Data Factory, I need to insert/update data from the Sql db to the GraphDb.

My thinking is that I need to first transform the data to json and from there insert it into the GraphDb. Is this the way to go? Are there any other ways?

Thank you.

Marius B.
  • 478
  • 6
  • 18
  • 1
    https://stackoverflow.com/questions/44143557/how-can-we-create-azures-data-factory-pipeline-with-cosoms-db-with-graph-api – silent Oct 25 '19 at 10:13
  • So it seems nothing changed from 2017. Thank you for the link @silent. – Marius B. Oct 25 '19 at 10:52
  • Yes, it does, thank you. So your suggestion is to use a bulk insert. I was thinking of also using the SQL API, since it works on the graphdb. – Marius B. Oct 28 '19 at 06:45

1 Answers1

3

1.Based on the ADF copy activity connector and the thread: How can we create Azure's Data Factory pipeline with Cosoms DB (with Graph API) as data sink ? mentioned by @silent,Cosmos db graph api connector is not supported in ADF so far. You could vote up this feature in this feedback link which is updated at April 12, 2019.

2.Cosmos db migration tool isn't a supported import tool for Gremlin API accounts at this time. Please see this link:https://learn.microsoft.com/en-us/azure/cosmos-db/import-data

3.You could get an idea of graph bulk executor .NET library now.This is the sample application:git clone https://github.com/Azure-Samples/azure-cosmosdb-graph-bulkexecutor-dotnet-getting-started.gi

Jay Gong
  • 23,163
  • 2
  • 27
  • 32
  • How would you suggest I integrate the bulk executor in the data factory, through an Azure function? – Marius B. Oct 28 '19 at 07:01
  • 1
    @MariusB. Yes,the bulk connector is just usage of sdk which could be done anywhere. Surely,you could invoke it in azure function activity. Any issues, you could post here and i will help you. – Jay Gong Oct 28 '19 at 07:07
  • I see there are two options : 1. To use the bulk executor you suggested or 2. to use the SQL API(since it also works on the graphdb). Do you save any insights about pros/cons(speed/cost and so on)? – Marius B. Oct 28 '19 at 07:15
  • @MariusB. i have no idea about sql api works on the graphdb, sql api and graph api are different in cosmos db,aren't them? Any links i need to sync? – Jay Gong Oct 28 '19 at 07:20
  • https://stackoverflow.com/questions/47045818/cosmosdb-creation-api-option-sql-vs-graph , this seems to be one of them, I also saw this on other sources. – Marius B. Oct 28 '19 at 07:26
  • Just made a small test, and managed to insert data from the SQL Db into the CosmodDb graph with SQL API. – Marius B. Oct 28 '19 at 08:03
  • @MariusB. Any issues? – Jay Gong Oct 28 '19 at 08:04
  • No issues,maybe I'm wrong, but I think SQL API is suited only for simple scenarios( when gettings data from SQL Db to GraphDb). – Marius B. Oct 28 '19 at 08:51
  • 1
    @MariusB. Agreed, graph db must be better supported by graph API. – Jay Gong Oct 28 '19 at 08:54