0

The quickest method I have found is to just convert the ItemPaged object to a list using list() and then I'm able to manipulate/extract using a Pandas DataFrame. However, if I have millions of results, the process can be quite time-consuming, especially if I only want every nth result over a certain time-frame, for instance. Typically, I would have to query the entire time-frame and then re-loop to only obtain every nth element. Does anyone know a more efficient way to use query_entities OR how to more efficiently return every nth item from ItemPaged or more explicitly from table.query_entities? Portion of my code below:

connection_string = "connection string here"
service = TableServiceClient.from_connection_string(conn_str=connection_string)
table_string = ""
table = service.get_table_client(table_string)
entities = table.query_entities(filter, select, etc.)
results = pd.DataFrame(list(entities))
dylan_mayes
  • 1

1 Answers1

0

Does anyone know a more efficient way to use query_entities OR how to more efficiently return every nth item from ItemPaged or more explicitly from table.query_entities?

After reproducing from my end, one of the ways to achieve your requirement using get_entity() instead of query_entities(). Below is the complete code that worked for me.

entity = tableClient.get_entity(partition_key='<PARTITION_KEY>', row_key='<ROW_KEY>')
print("Results using get_entity :")
print(format(entity))

RESULTS:

enter image description here

SwethaKandikonda
  • 7,513
  • 2
  • 4
  • 18
  • Yes, this is another option if you want to get just one entity or use this to loop and get multiple entities. In my case, I am querying to find specific RowKey values when other columns equal a certain value, so this will not work, unless there is a workaround that I am not aware of. Thanks for your help however. – dylan_mayes Jan 31 '23 at 20:50
  • @dylan_mayes, Can you add a sample of what you are trying to find – SwethaKandikonda Feb 01 '23 at 02:34
  • it would be something like filter="PartitionKey eq 1 and RowKey ge 2022-01-01 00:00:00 and RowKey lt 2022-01-31 00:00:00", select='Different column name' that I would typically put into query_entities function. I do not know the exact RowKey, so I cannot query for the specific RowKey that is needed in get_entity. – dylan_mayes Feb 02 '23 at 16:08