Dynamic table storage name

Question

is it possible to have a dynamic table storage table name in a data factory input?

Everything what I find is with a fix name

"properties": {
 "type": "AzureTable",
 "linkedServiceName": "StorageLinkedService",
 "typeProperties": {
     "tableName": "MyTable"
 },

The situation is that I want to copy the data of a dynamic list of tables into a sql database.

score 1 · Answer 1 · answered Oct 29 '19 at 10:06

You can parameterize the table name in Azure Data Factory v2 for an Azure Table Storage dataset.

I created this manually in the authoring tool, but the corresponding (exported) arm template looks like this:

"parameters": {
    "table": {
        "type": "string"
    }
},
"annotations": [],
"type": "AzureTable",
"schema": [],
"typeProperties": {
    "tableName": {
        "value": "@dataset().table",
        "type": "Expression"
    }
}

score 0 · Answer 2 · answered Mar 13 '17 at 15:33

0

I believe you will have to create separate Datasets with static table names. Dynamic table names are not possible!!

answered Mar 13 '17 at 15:33

Nimesh Parikh

11
2

score 0 · Answer 3 · answered Mar 14 '17 at 15:34

Sadly not!

I've just tested this is I was hoping that you'd be able to inject values from the partition by clause. However, this is only possible in file and folder names it seems. Not for tables.

See below screen shot.

Not allowed according to the portal, even though it deploys without failing validation!!

I've just created a MS connect feedback item for it, link below.

https://feedback.azure.com/forums/270578-data-factory/suggestions/18614509-support-the-partition-by-clause-for-azure-table-da

Go vote for it to be added to ADF functionality.

score 0 · Answer 4 · answered May 17 '17 at 10:39

If you use .NET API then you can achieve it.

using Microsoft.Azure.Management.DataFactories nuget package

var dataFactoryClient = new DataFactoryClient {
    ResourceGroupName = "",
    DataFactoryName = ""
}

var client = dataFactoryClient.GetDataFactoryManagementClient();

client.Datasets.CreateOrUpdate(ResourceGroupName, DataFactoryName,
                new DatasetCreateOrUpdateParameters
                {
                    Dataset = new Dataset
                    {
                        Name = datasetName,
                        Properties = new DatasetProperties
                        {
                            LinkedServiceName = linkedService,
                            TypeProperties = new AzureTableDataset
                            {
                                TableName = tableName //append current date

                            },
                            External = external,
                            Availability = availability,
                            Structure = //DataElements
                            Policy = new Policy {}

                        }
                    }
                });

You can also access pipelines, activities, linkedservices using this API and customize them as per the need.

You can use WebJob to deploy this DataFactory.

Dynamic table storage name

4 Answers4