How many IO bound async tasks should I fire at once?

Question

I am firing commands to IP cameras, about thousands of them, The code and network requests are in ConnectCam method. I use the below pattern to fire them all at once using Task.WhenAny(). Does this have any potential bottlenecks? Should I throttle the requests?

List<Task> tasks = deviceList.Select(x => ConnectCamAsync(ct, x.IpAddress, x.UserName, x.PassWord)).ToList();
       var orgCount = tasks.Count();
           while (tasks.Count() > 0)
            {
                ct.ThrowIfCancellationRequested();
                try
                {
                    var firstFinishedTask = await Task.WhenAny(tasks);

                    tasks.Remove(firstFinishedTask);

                    await firstFinishedTask;
                }
                finally
                {
                    UpdateProgress(0, orgCount, orgCount - tasks.Count());
                }
            }

There is only one way to answer this and that is to test it. It will depend on too many factors, many of them probably unknown until you discover them, to be able to provide a solid answer here. — Lasse V. Karlsen, Jan 28 '19 at 12:27
The answer to this question relates to many factors, including the buffering/queueing/memory management of all intermediate network devices. If you hit your network too hard, it's entirely possible that upstream devices will drop connections. What's the limit? You need to find that out yourself. — spender, Jan 28 '19 at 12:28
Also, reaching out to thousands of separate addresses will expose you to the risk of port starvation if you use more than your max ephemeral ports before the TIME_WAIT period of the connections. IIRC that's ~16k connections every 2 mins. — spender, Jan 28 '19 at 12:32
Thanks, Guys, Can you guys suggest a way to achieve optimal performance, Its not possible for me to test on thousands of devices. Can we have a safe guess around the number which can ensure safety at some expense to performance. — Manish Jha, Jan 28 '19 at 12:53
What's the point of `await firstFinishedTask;` meant to be? By definition, you know it's already complete, or else `WhenAny` is completely broken. — Damien_The_Unbeliever, Jan 28 '19 at 13:24
The thing is that `async` doesn't mean that you'll get more job done. It means that you can start more jobs than in a synchronous way. Just measure the performance — VMAtm, Jan 28 '19 at 18:36

score 1 · Accepted Answer · edited Jan 28 '19 at 14:20

1

If you open thousands of connections at the sametime, of course there will be performance bottlenecks. I don't think that this much request can be handled without limitation.

You can consider to use SemaphoreSlim

edited Jan 28 '19 at 14:20

Fildor

14,510
4
35
67

answered Jan 28 '19 at 12:30

Derviş Kayımbaşıoğlu

28,492
4
50
72

How many IO bound async tasks should I fire at once?

1 Answers1