1

I am moving data from Azure Table Storage (source) to Azure Table Storage (Destination)

I have lets say 15 columns in my source but I only want 7 of those in my destination. How can I, or what is the best way to map or query that. I have done some reading into OData queries but that hasn't helped. I am assuming I have to query something similar to a SQL select statement in the copy activity in ADF .

for example: this is the error I get

enter image description here

Thanks in advance. Please ask for clarification if needed.

gumdrop
  • 35
  • 1
  • 9

1 Answers1

0

I have lets say 15 columns in my source but I only want 7 of those in my destination. How can I, or what is the best way to map or query that.

You would use Query Projection for that and specify the attributes you want to import using $select OData query parameter.

For example if your table has the following attributes: A1, A2, A3, A4, A5, A6, A7, B1, B2, B3, B4, B5, B6 and B7 and you only want to fetch attributes A1, A2 and A3.

In this case your $select query parameter would be like:

$select=A1,A2,A3

Please see this link for more details: https://learn.microsoft.com/en-us/rest/api/storageservices/querying-tables-and-entities#supported-query-options.

Gaurav Mantri
  • 128,066
  • 12
  • 206
  • 241
  • Hi, I was doing exactly the same like you are suggesting, but im getting an error. I have edited my question above and added a snip of my error. please refer – gumdrop Oct 29 '21 at 02:32
  • Thanks for sharing more details. I remember seeing a similar question sometime back and if I recall correctly, ADF does not support query projection. You will need to fetch all attributes and do the query projection manually. – Gaurav Mantri Oct 29 '21 at 02:42
  • Thank you, when you say "do the query projection manually" what do you mean by that? Where else can I do query projection? (apologies, im very new to table storage) – gumdrop Oct 29 '21 at 02:44
  • And I am new to ADF :). I am guessing that somewhere in your ADF pipeline you will need to tell that you only want to import certain attributes. That's what I meant by doing the query projection manually. HTH. – Gaurav Mantri Oct 29 '21 at 02:50