Alternative to yielding in an IAsyncEnumerable

Question

I'm trying to retrieve data that is paginated using the new AsyncEnumerables in C# 8.0. Back in the synchronous IEnumerable world the code would look something like this:

private IEnumerable<S3Object> Example(S3FilesRequest requestData)
    {
        var request = new ListObjectsV2Request()
        {
            BucketName = requestData.Bucket,
            Prefix = requestData.KeyPrefix
        };

        ListObjectsV2Response response;
        do
        {
            response = this.client.ListObjectsV2Async(request).Result;
            foreach (var s3Obj in response.S3Objects)
                yield return s3Obj;
            request.ContinuationToken = response.NextContinuationToken;
        } while (response.IsTruncated);
    }
    // Not tested - can probably be more succinct (using TakeWhile and SelectMany) but you get the idea

Using IAsyncEnumerable I think that it would look something like this:

private IAsyncEnumerable<S3Object> listObjects(S3FilesRequest requestData)
        => AsyncEnumerable.CreateEnumerable(() =>
        {
            var request = new ListObjectsV2Request()
            {
                BucketName = requestData.Bucket,
                Prefix = requestData.KeyPrefix
            };
            ListObjectsV2Response response = null;
            List<S3Object>.Enumerator enumerator = default;
            S3Object current = null;
            bool movedNext = false;

            return AsyncEnumerable.CreateEnumerator<S3Object>(async c =>
            {
                if (response == null || !movedNext)
                {
                    enumerator.Dispose();
                    response = await this.client.ListObjectsV2Async(request, c);
                    enumerator = response.S3Objects.GetEnumerator();
                }

                movedNext = enumerator.MoveNext();
                current = enumerator.Current;

                request.ContinuationToken = response.NextContinuationToken;
                return movedNext || response.IsTruncated;
            }, () => current, enumerator.Dispose);
        });

This seems a bit long winded, but without the yield keyword is there a better way?

Thanks,

Joe

EDIT:

This was caused by ReSharper silently installing the package System.Interactive.Async whilst upgrading to C# 8. I uninstalled it and followed Stephen's advice.

For Google:

CS0518  Predefined type 'System.Runtime.CompilerServices.AsyncIteratorMethodBuilder' is not defined or imported
CS0656  Missing compiler required member 'System.Runtime.CompilerServices.AsyncIteratorMethodBuilder.Create'
CS0656  Missing compiler required member 'System.Collections.Generic.IAsyncEnumerable`1.GetAsyncEnumerator'
CS0656  Missing compiler required member 'System.Collections.Generic.IAsyncEnumerator`1.MoveNextAsync'
CS0656  Missing compiler required member 'System.IAsyncDisposable.DisposeAsync'
CS0656  Missing compiler required member 'System.Threading.Tasks.ValueTask`1..ctor' 
CS0656  Missing compiler required member 'System.Threading.Tasks.ValueTask..ctor'
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.GetResult'cs
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.GetStatus'cs
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.get_Version'cs 
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.OnCompleted'cs
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.Reset' 
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.SetExceptioncs 
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1.SetResult'cs 
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.IValueTaskSource`1.GetResult' 
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.IValueTaskSource`1.GetStatus' 
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.IValueTaskSource`1.OnCompleted'
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.IValueTaskSource.GetResult' 
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.IValueTaskSource.GetStatus' 
CS0656  Missing compiler required member 'System.Threading.Tasks.Sources.IValueTaskSource.OnCompleted'

By the way, why was this question down voted? – JoeS Jul 06 '19 at 22:13 — JoeS, Jul 06 '19 at 22:13

score 5 · Accepted Answer · answered Jul 06 '19 at 01:52

5

If you're using C# 8.0, then you can use yield with async just fine:

private async IAsyncEnumerable<S3Object> Example(S3FilesRequest requestData)
{
  var request = new ListObjectsV2Request()
  {
    BucketName = requestData.Bucket,
    Prefix = requestData.KeyPrefix
  };

  ListObjectsV2Response response;
  do
  {
    response = await this.client.ListObjectsV2Async(request);
    foreach (var s3Obj in response.S3Objects)
      yield return s3Obj;
    request.ContinuationToken = response.NextContinuationToken;
  } while (response.IsTruncated);
}

(and asynchronous LINQ methods like TakeWhile and SelectMany are in the System.Linq.Async package)

answered Jul 06 '19 at 01:52

Stephen Cleary

437,863
77
675
810

Thanks Stephen, that package will be very useful. However the code above is basically what I initially wrote. If I try to use it though I don't get any red underlines in VS but I get lots of build errors like this: `CS0656 Missing compiler required member System.Threading.Tasks.ValueTask1..ctor'` `CS0518 Predefined type 'System.Runtime.CompilerServices.AsyncIteratorMethodBuilder' is not defined or imported` etc. Which makes me think that I might have broken my project during the upgrade to C# 8 or I'm missing some core packages, I'll investigate further. – JoeS Jul 06 '19 at 21:51
3

Solved it, when I used IAsyncEnumerable ReSharper offered to upgrade my language version, it also silently installed the package System.Interactive.Async. I've uninstalled it and installed System.Linq.Async and now everything works. Thanks for your help. – JoeS Jul 06 '19 at 22:04

Alternative to yielding in an IAsyncEnumerable

1 Answers1