14

I'm really confused about async-awaits, pools and threads. The main problem starts with this question: "What can I do when I have to handle 10k socket I/O?" (aka The C10k Problem).

  • First, I tried to make a custom pooling architecture with threads that uses one main Queue and multiple Threads to process all incoming datas. It was a great experience about understanding thread-safety and multi-threading but thread is an overkill with async-await nowadays.
  • Later, I implemented a simple architecture with async-await but I can't understand why "The async and await keywords don't cause additional threads to be created." (from MSDN)? I think there must be some threads to do jobs like BackgroundWorker.
  • Finally, I implemented another architecture with ThreadPool and it looks like my first custom pooling.

Now, I think there should be someone else with me who confused about handling The C10k. My project is a dedicated (central) server for my game project that is hub/lobby server like MCSG's lobbies or COD's matchmaking servers. I'll do the login operations, game server command executions/queries and information serving (like version, patch).

Last part might be more specific about my project but I really need some good suggestions about real world solutions about multiple (heavy) data handling.

(Also yes, 1k-10k-100k connection handling depending on server hardware but this is a general question)

The key point: Choosing Between the Task Parallel Library and the ThreadPool (MSDN Blog)

[ADDITIONAL] Good (basic) things to read who wants to understand what are we talking about:

  1. Threads
  2. Async, Await
  3. ThreadPool
  4. BackgroundWorker
Thus Spoke Nomad
  • 2,372
  • 4
  • 17
  • 34
  • That quote about `await`-`async` has to be the most intentionally misleading thing I've ever seen in an official document. I remember looking through dozens of documents during the preview and they all made it sound like the pattern didn't use threads, they did some crazy in-thread code flow, but that's just not true. What they *mean* is that it doesn't *spawn new threads*, it uses the normal `Task.StartNew` thread pool. – Blindy Mar 13 '15 at 12:43
  • 1
    @Blindy You're wrong. async/await doesn't itself uses `Task.StartNew`. It won't create any thread or call `Task.StartNew` if you don't call it yourself. `await` keyword just sets up the continuation and that's it, no more. – Sriram Sakthivel Mar 13 '15 at 12:47
  • @Blindy What Sriram said. And in case of asynchronous I/O in particular, you only need a thread for the callback itself - this means that you can usually handle thousands of connections with just a couple of threads (usually around twice the amount of CPU cores - after all, if you're doing everything non-CPU asynchronously, your limit is 100% CPU, and adding threads doesn't help with throughput in that case). – Luaan Mar 13 '15 at 13:12

2 Answers2

4

async/await is roughly analogous to the "Serve many clients with each thread, and use asynchronous I/O and completion notification" approach in your referenced article.

While async and await by themselves do not cause any additional threads, they will make use of thread pool threads if an async method resumes on a thread pool context. Note that the async interaction with ThreadPool is highly optimized; it is very doubtful that you can use Thread or ThreadPool to get the same performance (with a reasonable time for development).

If you can, I'd recommend using an existing protocol - e.g., SignalR. This will greatly simplify your code, since there are many (many) pitfalls to writing your own TCP/IP protocol. SignalR can be self-hosted or hosted on ASP.NET.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • Let me ask one more question: `ThreadPool` or `Task` creates `Thread`s automatically _if needed_ right? Namely I only need to create a handling method and call `Task`s (StartNew etc) or `ThreadPool.QueueUserWorkItem`? – Thus Spoke Nomad Mar 13 '15 at 13:54
  • 1
    @MonoLightTech: `ThreadPool` creates threads as necessary. `Task` is a higher abstraction; thread pool threads have their own task queues (with work-stealing capabilities). Depending on how your library invokes your handler, you can either make the handler `async` directly or just use `Task.Run` to jump on an available thread pool thread. – Stephen Cleary Mar 13 '15 at 14:23
  • I guess it is to avoid deadlocks – Sergei Krivonos Jul 27 '17 at 13:38
4

No. If we use asynchronous programming pattern that .NET introduced in 4.5, in most of the cases we need not to create manual thread by us. The compiler does the difficult work that the developer used to do. Creating a new thread is costly, it takes time. Unless we need to control a thread, then “Task-based Asynchronous Pattern (TAP)” and “Task Parallel Library (TPL)” is good enough for asynchronous and parallel programming. TAP and TPL uses Task. In general Task uses the thread from ThreadPool(A thread pool is a collection of threads already created and maintained by .NET framework. If we use Task, most of the cases we need not to use thread pool directly. A thread can do many more useful things. You can read more about Thread Pooling

You can avoid performance bottlenecks and enhance the overall responsiveness of your application by using asynchronous programming. Asynchrony is essential for activities that are potentially blocking, such as when your application accesses the web. Access to a web resource sometimes is slow or delayed. If such an activity is blocked within a synchronous process, the entire application must wait. In an asynchronous process, the application can continue with other work that doesn't depend on the web resource until the potentially blocking task finishes.

Await is specifically designed to deal with something taking time, most typically an I/O request. Which traditionally was done with a callback when the I/O request was complete. Writing code that relies on these callbacks is quite difficult, await greatly simplifies it. Await just takes care of dealing with the delay, it doesn't otherwise do anything that a thread does. The await expression, what's at the right of the await keyword, is what gets the job done. You can use Async with any method that returns a Task. The XxxxAsync() methods are just precooked ones in the .NET framework for common operations that take time. Like downloading data from a web server.

I would recommend you to read Asynchronous Programming with Async and Await

Mohit S
  • 13,723
  • 6
  • 34
  • 69