0

I have a rest API that takes longer to populate complete result data (as it iterates the remove directory recursively which is time taking sometimes) during that time the client is keep waiting. once all the data is populated we are sending a response back to the client.

The problem with the above method is the client is keep waiting until all the data is populated.

is there a way in .net, the Rest API sends the data back to the client in chunks as the data keeps on populating? (without client to wait till whole data is generated).

Can you provide a sample of how to achieve the above requirement in .Net ?

KAPIL RATHORE
  • 1
  • 2
  • 1
    It's a bit broad to be answered accurately, but there are at least 2 ways to achieve that : First is not sending the whole data, but paginate it (the first N results or the results X to X + N). Second is to do asynchronous calls to avoid the client to hang – Cid Jan 31 '22 at 14:35
  • Are you looking for [the `PushStreamContent` type](https://learn.microsoft.com/en-us/previous-versions/aspnet/hh995285(v=vs.118))? [Stephen Cleary wrote about it back in 2016](https://blog.stephencleary.com/2016/10/async-pushstreamcontent.html). – Richard Deeming Jan 31 '22 at 15:00
  • I may be wrong but isn't IAsyncEnumerable used for this kind of thing? https://stackoverflow.com/questions/58876817/clarification-on-how-iasyncenumerable-works-with-asp-net-web-api – Neil Jan 31 '22 at 15:25
  • Does this answer your question? [Clarification on how IAsyncEnumerable works with ASP.NET Web API](https://stackoverflow.com/questions/58876817/clarification-on-how-iasyncenumerable-works-with-asp-net-web-api) – Neil Jan 31 '22 at 15:25

2 Answers2

1

It depends how long client should wait.

For long operations maybe it better to create more endpoints:

  1. endpoint to start operation, returns operation id
  2. endpoint for cheking operation status for given id
    returns response:
    • operation status: completed, in-progress
    • jobs (chunks), chunk:id1 - completed, chunk:id2 - in-progress
  3. endpoint for retrieving jobs (chunks) results - for given operation id and chunk id
apocalypse
  • 5,764
  • 9
  • 47
  • 95
  • I am new to asp.net any sample code can you please point to? – KAPIL RATHORE Jan 31 '22 at 16:16
  • just create 3 endpoints: api/v1/operations/start, api/v1/operations/status, api/v1/operations/getchunk. If you dont understand how REST Api works, you need to start from some tutorial... – apocalypse Jan 31 '22 at 16:52
-1

The solution depends if the client needs the data all at once or not.

If data is needed in parts then you should use pagination (in my opinion this is the correct way).

If data is needed all at once then you should make 2 end-point (one for starting the job in background and one for downloading data once it's done) and a background worker (Web Jobs or Azure Functions that could store the result in Blob Storage or Database) witch will notify the client when the process is finished.

dobre.b
  • 64
  • 1
  • 9