0

In C++, I've read some tutorials to create a server which can accept connections from multiple clients. They suggest using async socket, but i don't really know why we should choose async over none-blocking mode. And what's about the ideas that use multi-threading? is it better than using async socket? Thanks!!

Some programmer dude
  • 400,186
  • 35
  • 402
  • 621
Kingfisher Phuoc
  • 8,052
  • 9
  • 46
  • 86
  • It depends much on the design of the application as a whole. If it's already async/event driven then using async sockets may be best, if it's already have its own event loop or polling then using non-blocking sockets with polling might be best, and if you want to receive connections and then "forget" about them (much like a web-server) then threads will work well. – Some programmer dude Jun 27 '12 at 05:46
  • As an addendum to my comment, about threads... They can be good when the connections are independent of each other, and don't need to communicate with other connections and/or the main thread. – Some programmer dude Jun 27 '12 at 05:54
  • If you go threading, then you can take it a step further by using IOCP to reduce the number of threads needed while increasing the number of connections you can handle. Communicating with the main thread is not a reason to avoid threading the connctions. Most of the socket logic doesn't need to access the main thread, so it would benefit from threading. Main thread accesses should be small operations, so would not slow down the threads very much. – Remy Lebeau Jun 27 '12 at 23:37

2 Answers2

1

Since you're requesting a solution in C++, boost asio is imo the best async io library there is.

I assume you're talking about the "one thread per client" solution when refering to "multi-threading", which is generally a very bad idea for servers who expect many clients in a short time frame or connected at the same time. Threads are way to resource consuming for this use, plus you have to take care of mutual exclusion, which in combination with blocking calls can drive you into deadlocks very fast. And thats the least worst of what you can run into.

Additionally on that, it's very easy for an attacker to exploit your server to stuck. You will spend much time on trying to design your code so that this will be avoided, which leads you into having an unreadable, hard to update and error phrone code.

In boost.asio the specified thread(s) ( those who call io_service::run ) will only do work when there is actually work to do, directly leading you into the object assigned to the task. So technically async is also blocking, with the difference that only the scheduler waits for work to do, while those functions you add work with ( connect, send, receive, ... ) will return immediately.

Andy
  • 176
  • 1
  • 1
  • 10
  • Why should it be any easier to stick a one-thread-per-client server than any other kind? If one client-handler thread does get stuck, so what? If the one thread on an async server gets stuck, that's goodbye to all clients. Async client code is a state-machine instead of simple in-line calls that are easy to make. One thread per client does not mean 'unreadable, hard to update and error phrone code', it's just in-line code where you CAN make blocking calls. Iif you need a 10 second pause, use Sleep(10000) -no problem. Try that in an async handler! – Martin James Jun 27 '12 at 07:55
  • @MartinJames: A one-thread-per-client server is also a one-stack-per-client server. Default size is a megabyte, so 2000 connections are 2GB. Even assuming that there's enough physical RAM for this, it's a cache nightmare, and on a 32bit OS, you run out of address space -- open 2000 connections, and do nothing else... BANG you're dead. I'm not even talking of the scheduler load for handling thousands of threads. On a single threaded state machine, the same would be something like 10-20kB of data plus maybe another 10-20kB inside the IOCP. As long as nothing happens on the wire, zero action. – Damon Jun 27 '12 at 08:07
  • In general I agree that an ASIO/IOCP server with state-machines is much more scaleable, and is also more complex. Last time I used IOCP, you had to add a sequence number to every completion instance received in its linked socket object to prevent out-of-order handling of the buffers by pooled threads. For one-stack-per-client, 1 MB is too large - set it to 128K. There does not have to be physical RAM - user stacks are swappable. The scheduler load for thousands of non-ready threads is zero - there is nothing to schedule - so as long as nothing happens on the wire, zero action. – Martin James Jun 27 '12 at 14:46
  • @Damon I just checked on my box. It has 1190 threads now, just from ordinary loaded apps. CPU load 2% and 1% of that seems to be 'vlc.exe', even though I'm not playing any music/video at the moment, (must Kaspersy again, just in case..). 'NT kernel and system' has 238 threads on its own. kaspersky has 71. – Martin James Jun 27 '12 at 14:57
  • @MartinJames: Note that 2000 threads in, say, 50 or 100 processes is not the same as 2000 threads in one process, as far as address space goes. This is independent on committing stack memory (i.e. using physical memory). About "nothing to schedule", I digress. A thread is either "ready", then it's on the scheduler's ready list, or it is not ready (then it's on the not ready list). Ok, there are other conditions (e.g. uninterruptible sleep) too, but let's not make it too complicated. When something happens on a socket, the scheduler must walk the not-ready list, find and remove the thread, and – Damon Jun 27 '12 at 18:26
  • move it to the ready list so it can be scheduled (and, it gives the thread a temporary priority boost fwiw). Walking over the not-ready list to find the correct thread is O(N), so more threads are certainly more work whenever something happens. – Damon Jun 27 '12 at 18:27
  • 1
    There is no not-ready list. There is an array[priority] of queues of ready threads and after any hardware/software interrupt that changes the content of this struct, the scheduler takes [no of cores] threads from the heads of the queues by polling them from the highest prio. down. non-ready threads are on a queue too - against the signal they are waiting on. It desn't matter how long the object-wait queues are because the OS only needs the head entry. – Martin James Jun 27 '12 at 20:02
  • 1
    ..and, on the infamous 'tick/quantum' interrupt, the OS only needs to rotate the tail of the ready queues to the head, so implementing the 'round-robin' scheduling that some developers seem to think is the norm :( If you want to try this, write a simple app that merely creates, say, 3000 threads that sleep (INFINITE). Run it, then continue to use your box and see just how unresponsive it is. The result will be that it operates normally. – Martin James Jun 27 '12 at 20:06
1

I'll assume you're talking TCP and not UDP. I definitely recommend skipping async sockets, those are favored by Microsoft and supporters but are not portable. Instead use the vanilla stuff: here's an example with server and client.

Jonas Byström
  • 25,316
  • 23
  • 100
  • 147