Error while reading data from azure table storage

Question

I'm writing a code to read values from a table storage. The code is similar to printing nodes in a tree level by level.

eg:

root

Level1child1 -> Level1child2 -> Level1child3

string tablename = "<table-name">;
string storageAccountName = "<storage-account-name";

var baseurl = @$"https://{storageAccountName}.table.core.windows.net/{tableName}()";
var sastoken = getAccountSASToken();
string filter = @"&$filter=PartitionKey%20eq%20'123'%20and%20RowKey%20eq%20'abc'";
baseurl = $"{baseurl}{sastoken}{filter}";
var data = HttpHelper.GetForOData(baseurl);
var responseData = data.Data.Replace(".", "_");
var odata = JsonConvert.DeserializeObject<ODataResponse>(responseData);
Queue<int> strQ = new Queue<int>();
Console.WriteLine(odata.value[0].Email);
strQ.Enqueue(odata.value[0].TreeNodeID);

while (strQ.Any())
{
    var url = @$"https://{storageAccountName}.table.core.windows.net/{tableName}()";
    var token = _tableStorageRepository.GetAccountSASToken();
    filter = @"&$filter=ParentNodeId%20eq%20" + strQ.Peek();
    url = $"{url}{token}{filter}";
    data = HttpHelper.GetForOData(url);
    responseData = data.Data.Replace(".", "_");
    odata = JsonConvert.DeserializeObject<ODataResponse>(responseData);

    foreach (var m in odata?.value)
    {
        Console.WriteLine(m.Email);
        strQ.Enqueue(m.TreeNodeID);
    }
    strQ.Dequeue();
}

    public class ODataResponse
    {
        public string odata_metadata { get; set; }
        public List<ODatavalue> value { get; set; }
    }
    public class ODatavalue
    {
        public string odata_type { get; set; }
        public string odata_id { get; set; }
        public string odata_etag { get; set; }
        public string odata_editLink { get; set; }            
        public string RowKey { get; set; }            
        public string Email { get; set; }            
        public int ParentNodeID { get; set; }            
        public int TreeNodeID { get; set; }

    }

Code for HttpHelper class: https://github.com/xyz92/httphelper/blob/master/HttpHelper.cs

The first time when I ran this code, it only printed root node.

The second time when I ran this code, it printed root node and Level1child1 node.

For the next runs, it printed root node, Level1child1 node & Level1child2 node.

The last node Level1child3 node is getting printed very rarely on some runs.

What am I missing in this code?

UPDATE:

Sample responseData:

{
    "odata_metadata": "https://<storage-account-name>_table_core_windows_net/$metadata#<table-name>",
    "value": [{
            "odata_type": "<storage-account-name>_<table-name>",
            "odata_id": "https://<storage-account-name>_table_core_windows_net/<table-name>(PartitionKey='123',RowKey='abc')",
            "odata_etag": "W/\"datetime'2020-09-01T16%3A34%3A21_3342187Z'\"",
            "odata_editLink": "<table-name>(PartitionKey='123',RowKey='abc')",
            "PartitionKey": "123",
            "RowKey": "abc",
            "Timestamp@odata_type": "Edm_DateTime",
            "Timestamp": "2020-09-01T16:34:21_3342187Z",
            "Email": "email",
            "ParentNodeID": 1,
            "TreeNodeID": 2
        }
    ]
}

Table columns:

Sample data in Table:

Sample outputs while running code:

@user989988 Could you please tell me how you define `ODataResponse` class. Besides, could you please provide the screenshot of some run time result. — Jim Xu, Sep 02 '20 at 02:26

Jim Xu · Answer 1 · 2020-09-03T01:38:00.920

According to my test, I cannot reproduce your issue in my environment. My test is as below

Table columns:

Sample data in table

Code I use your HttpHelper class to send request

class Program
    {
        static async Task Main(string[] args)
        {
            string storageAccountName = "andyprivate" ;
            string tableName = "test";

            var baseurl = @$"https://{storageAccountName}.table.core.windows.net/{tableName}()";
            var sastoken = "";
            string filter = @"&$filter=PartitionKey%20eq%20'123'%20and%20RowKey%20eq%20'abc'";
            baseurl = $"{baseurl}{sastoken}{filter}";
            var data = HttpHelper.GetForOData(baseurl);
            var responseData = data.Data.Replace(".", "_");
            var odata = JsonConvert.DeserializeObject<ODataResponse>(responseData);
            Queue<int> strQ = new Queue<int>();
            Console.WriteLine("----root----");
            Console.WriteLine(odata.value[0].Email);
            strQ.Enqueue(odata.value[0].TreeNodeID);
            while (strQ.Any())
            {
                int i = 0;
               
                var url = @$"https://{storageAccountName}.table.core.windows.net/{tableName}()";
                var token = "";
                filter = @$"&$filter=ParentNodeID eq {strQ.Peek()}";
                url = $"{ url}{token}{filter}";
                data = HttpHelper.GetForOData(url);
                responseData = data.Data.Replace(".", "_");
                odata = JsonConvert.DeserializeObject<ODataResponse>(responseData);
                Console.WriteLine($"----TreeNode{strQ.Peek()}----");
                foreach (var m in odata?.value)
                {
                    if (i == 0) {
                        strQ.Enqueue(m.TreeNodeID);
                        i = 1;
                    }                   
                    Console.WriteLine(m.Email);
                    
                }
                strQ.Dequeue();
            }
            Console.ReadLine();
        }

      
    }
    public class ODataResponse
    {
        public string odata_metadata { get; set; }
        public List<ODatavalue> value { get; set; }
    }
    public class ODatavalue
    {
        public string odata_type { get; set; }
        public string odata_id { get; set; }
        public string odata_etag { get; set; }
        public string odata_editLink { get; set; }
        public string RowKey { get; set; }
        public string Email { get; set; }
        public int ParentNodeID { get; set; }
        public int TreeNodeID { get; set; }

    }

Update

My test code

Rest API (I use your HttpHelper class to send request)

class Program
    {
        static async Task Main(string[] args)
        {
            string storageAccountName = "andyprivate" ;
            string tableName = "test";

            var baseurl = @$"https://{storageAccountName}.table.core.windows.net/{tableName}()";
            var sastoken = "";
            string filter = @"&$filter=PartitionKey%20eq%20'123'%20and%20RowKey%20eq%20'abc'";
            baseurl = $"{baseurl}{sastoken}{filter}";
            var data = HttpHelper.GetForOData(baseurl);
            var responseData = data.Data.Replace(".", "_");
            var odata = JsonConvert.DeserializeObject<ODataResponse>(responseData);
            Queue<int> strQ = new Queue<int>();
            Console.WriteLine("----root----");
            Console.WriteLine(odata.value[0].Email);
            strQ.Enqueue(odata.value[0].TreeNodeID);
            while (strQ.Any())
            {
                int i = 0;

                var url = @$"https://{storageAccountName}.table.core.windows.net/{tableName}()";
                var token = "";
                filter = @$"&$filter=ParentNodeID eq {strQ.Peek()}";
                url = $"{ url}{token}{filter}";
                data = HttpHelper.GetForOData(url);
                responseData = data.Data.Replace(".", "_");
                odata = JsonConvert.DeserializeObject<ODataResponse>(responseData);
                Console.WriteLine($"----TreeNode{strQ.Peek()}----");
                foreach (var m in odata?.value)
                {
                    if (i == 0) {
                        strQ.Enqueue(m.TreeNodeID);
                        i = 1;
                    }                   
                    Console.WriteLine(m.Email);

                }
                strQ.Dequeue();
            }
            Console.ReadLine();
        }


    }
    public class ODataResponse
    {
        public string odata_metadata { get; set; }
        public List<ODatavalue> value { get; set; }
    }
    public class ODatavalue
    {
        public string odata_type { get; set; }
        public string odata_id { get; set; }
        public string odata_etag { get; set; }
        public string odata_editLink { get; set; }
        public string RowKey { get; set; }
        public string Email { get; set; }
        public int ParentNodeID { get; set; }
        public int TreeNodeID { get; set; }

    }

SDK. I use the package Microsoft.Azure.Cosmos.Table

string storageAccountName = "andyprivate";
            string accountKey ="";
            string tableName = "test";
            CloudStorageAccount storageAccount = new CloudStorageAccount(new StorageCredentials(storageAccountName, accountKey), true);
            CloudTableClient tableClient = storageAccount.CreateCloudTableClient(new TableClientConfiguration());
            CloudTable table = tableClient.GetTableReference(tableName);
            TableOperation retrieveOperation = TableOperation.Retrieve<CustomEntity>("123", "abc");
            TableResult result = await table.ExecuteAsync(retrieveOperation);
            CustomEntity entity = result.Result as CustomEntity;
            Queue<int> strQ = new Queue<int>();
            Console.WriteLine("----root----");
            Console.WriteLine(entity.Email);
            strQ.Enqueue(entity.TreeNodeID);
            while (strQ.Any()) {
                int i = 0;
                TableQuery<CustomEntity> query = new TableQuery<CustomEntity>()
                    .Where(                      
                          TableQuery.GenerateFilterConditionForInt("ParentNodeID", QueryComparisons.Equal,strQ.Peek())
                    );
                Console.WriteLine($"----TreeNode{strQ.Peek()}----");
                foreach (CustomEntity m in table.ExecuteQuery(query) ) {
                    Console.WriteLine(m.Email);
                    if (i == 0) {
                        strQ.Enqueue(m.TreeNodeID);
                        i = 1;
                    }
                } 
                strQ.Dequeue();
            }

Could you please share your code which I can run/test? Thank you! — user989988, Sep 02 '20 at 22:13
Thank you! Let me try this code although I don’t see any difference from my code. Just to confirm - this works even of there is one more level in the tree correct? — user989988, Sep 03 '20 at 15:02
@user989988 Do you have any other concerns? If you have no other concerns, could you please accept it as an answer? — Jim Xu, Sep 04 '20 at 04:55
I need to use filter query similar to my code instead of LINQ query. Is that possible whiel using package Microsoft.Azure.Cosmos.Table? — user989988, Sep 06 '20 at 19:29
@user989988 Hi. If you want to define the filter condition, we can create `TableQuery` class : https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.cosmos.table.tablequery?view=azure-dotnet — Jim Xu, Sep 07 '20 at 01:06

Error while reading data from azure table storage

1 Answers1