I am using the Azure.Data.Tables package & TableClient.QueryAsync() method to get the query result. I wants the result to use it for pagination. I came across this code in https://learn.microsoft.com/en-us/dotnet/api/azure.data.tables.tableclient.queryasync?view=azure-dotnet
Pageable<T> pageable = client.QueryAsync<T>(filter : value, maxPerPage : 10);
What are the changes should I make ?
Thanks in Advance !! :)
Here is a snippet from our samples.
AsyncPageable<TableEntity> queryResultsMaxPerPage = tableClient.QueryAsync<TableEntity>(filter: $"PartitionKey eq '{partitionKey}'", maxPerPage: 10);
await foreach (Page<TableEntity> page in queryResultsMaxPerPage.AsPages())
{
Console.WriteLine("This is a new page!");
foreach (TableEntity qEntity in page.Values)
{
Console.WriteLine($"# of {qEntity.GetString("Product")} inventoried: {qEntity.GetInt32("Quantity")}");
}
}
I solved this by creating an extension method for IAsyncEnumerable which will return the first item of an IAsyncEnumerable. After calling AsPages() on the AsyncPageable<T> query you have an IAsyncEnumerable<Page<T>> from which you can get the first page with this method.
static class IAsyncEnumerableExtensions
{
public static async Task<T?> FirstOrDefault<T>(this IAsyncEnumerable<T> asyncEnumerable)
{
await foreach (var item in asyncEnumerable)
{
return item;
}
return default;
}
}
The Page<T> object has a continuationToken property which can be passed to the AsPages(string) method, so you can get the next page with:
query.AsPages(continuationToken).FirstOrDefault();
In order to be able to get the previous page you will have to cache the continuationTokens, or for the first page just call query.AsPages().FirstOrDefault(); (without a continuationToken).
Check the documentation for more details (click through the parameter and return types).
I used this example as inspiration for the token caching. You can also look for examples using the Microsoft.Azure.Cosmos.Table library, here a similar concept is used with ContinuationTokens.
The key to achieve pagination is passing so-called ContinuationToken back and forth. When the extension method is defined as following:
public static class TableReader
{
public static async Task<Page<T>> GetPageAsync<T>(this TableClient client, string filter, int pageSize, string continuationToken)
where T: class, ITableEntity, new()
{
AsyncPageable<T> pageable = client.QueryAsync<T>(filter, pageSize);
await using (IAsyncEnumerator<Page<T>> enumerator = pageable.AsPages(continuationToken).GetAsyncEnumerator())
{
await enumerator.MoveNextAsync();
return enumerator.Current;
}
}
}
the return type of Page comprises both the data of single page and a token that can be used to obtain next page. The demonstration of the usage of this method could be:
string connectionString = "TODO: your connection string";
string tableName = "TODO: your table";
string filter = "TODO: your filter";
int pageSize = 1; //TODO: your page size
TableClient client = new TableClient(connectionString, tableName);
// continuationToken is used iterate through pages
string continuationToken = null;
while (true)
{
// take next page
Page<TableEntity> page = await client.GetPageAsync<TableEntity>(filter, pageSize, continuationToken);
foreach (TableEntity tableEntity in page.Values)
{
Console.WriteLine(tableEntity.RowKey);
}
Console.WriteLine("=== end of current page ===");
// if there are no more pages, we're done
if (page.ContinuationToken == null)
{
break;
}
// there are more pages => we can continue reading if we want
Console.WriteLine("> Press Y to load next page");
if (Console.ReadKey().Key.ToString().ToLower() != "y") { break; }
// cache the token to be used later to obtain next page
continuationToken = page.ContinuationToken;
}
Console.WriteLine("=== end of data ===");
Of course, you would use your own type implementing ITableEntity with additional properties instead of the basic TableEntity to parametrize the GetPageAsync method.
I can empathize with your question; I was banging my head against the wall on this for a while.
EDIT: This may fit your use case also: https://learn.microsoft.com/en-us/dotnet/azure/sdk/pagination#take-the-first-n-elements if your filter determines the lex ordering and a simple take count is used.
Here is a snippet from our samples.
Nope, this is NOT helpful, if anything it is determinantal in its current form. I cannot help but nit on this; I've spent days going over those "samples", reviewing the underlying source code, the tests themselves, and god knows how many times I have re-read MIGRATION GUIDE. While these samples would be somewhat high-level example of usage in a Console App perhaps as a Service Worker, however it doesn't give ANY guidance in where the lion share of APIs would be using this, i.e. repository pattern in a microservice. Don't even get me started on the TestEnvironment base class
Even reviewing the SDK guideline for Azure.* packages offer little of the HOW, just a generous helping of '...' Paging. That leaves devs back to square one and in my own experience I ended up with now more questions than answers.
Another unreal oversight is cancellation tokens, so I will address that elephant while I am here.
Now let's focus on answering the question with a functional example of what could be used in an application. I currently use this in my own production application:
// Background context: Class is a data provider class that gets videos from a video table of an ephemeral nature.
// Periodically the InitializeAsync is called to create the table client and instantiate a new table since the entire table is deleted after running the associated batch processing.
// This is specific to my use case and you can easily revert to
// having the TableClient specified via ctor injection instead with a DI Singleton registration.
public class VideoTable : IVideoTable
{
private readonly TableServiceClient service;
private readonly IOptionsMonitor<BotConfiguration> settings;
private readonly ILogger<VideoTable> logger;
private TableClient client;
// For mocking instantances
protected VideoTable() { }
public VideoTable(TableServiceClient service, IOptionsMonitor<BotConfiguration> settings, ILogger<VideoTable> logger)
{
logger.LogDebug("{Object} constructed", GetType().Name);
this.service = service;
this.settings = settings;
this.logger = logger;
}
public async Task InitalizeAsync(CancellationToken ct)
{
_ = await service.CreateTableIfNotExistsAsync(settings.CurrentValue.Storage.Table.Videos.Name, ct).Go(); // Note: to explain where this comes from: I got tired of writing '.ConfigureAwait(false);' so I made this extension method, simply remove them and you'll obtain the same functionality
client = service.GetTableClient(settings.CurrentValue.Storage.Table.Videos.Name);
}
public async IAsyncEnumerable<Page<VideoEntity>> GetVideosPaginatedAsync(int pageNumber, int perPageCount, [EnumeratorCancellation] CancellationToken ct)
{
var accumulator = 0;
var total = pageNumber * perPageCount; // say we want page 3 with 10 items per page we'll go to 30 here
var select = client.QueryAsync<VideoEntity>(maxPerPage: perPageCount, cancellationToken: ct);
do
{
await foreach (var page in select.AsPages().WithCancellation(ct)) //note under the hood the ConfigureAwait is already specified, no need to specifiy it again. Honorable mention you can specify a different cancellation source token and combine them if you don't want a single use token for cancellations
{
if (page.ContinuationToken is null || page.Values?.Count == 0)
{
yield break;
}
accumulator += page.Values.Count;
yield return page;
}
}
while (total > accumulator);
}
}
