Using IOCP with multiple listeners

Question

How do I setup IOCP sockets for multiple listeners (on different ports)? Every example I find online is just a single server - multiple clients example, and I don't understand if I supposed to create multiple IOCPs, or use only one for all listeners somehow?

For example, I've found this example on GitHub, which seems to be a slight modification of some code from Win 7 SDK, and this code creates a single completion port for a single listener socket:

g_hIOCP = CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 0);

for( DWORD dwCPU = 0; dwCPU < g_dwThreadCount; dwCPU++ )
{
    ...
    // associate worker thread with this iocp
    hThread = CreateThread(NULL, 0, WorkerThread, g_hIOCP, 0, &dwThreadId);
    ...
}

How do I use this same thread pool for multiple completion ports? Or do I somehow only need one completion port for all listeners sockets?

I do not know if it can help, but I have this example saved: http://www.serverframework.com/asynchronousevents/2012/03/windows-8-registered-io---multi-threaded-rio-iocp-udp-example -server.html — lsalamon, Mar 24 '17 at 14:03

score 1 · Accepted Answer · answered Mar 24 '17 at 13:41

you need only one completion port for all listeners sockets. and several threads (thread pool) which will be listen on this completion port. also possible not direct yourself create completion port and thread pool, but delegate this task to system (ntdll). this is can be done by using BindIoCompletionCallback or CreateThreadpoolIo

so we have - single completion port, several working threads listening to this port (usual number of threads == number of processor cores) and multiple files(sockets) associated with this completion port via BindIoCompletionCallback or CreateThreadpoolIo or CreateIOCompletionPort (ugly logic - compare with ZwSetInformationFile(..FileCompletionInformation) which is internally used)

score 1 · Answer 2 · answered Mar 24 '17 at 15:13

When initializing the server system, issue an AcceptEx() call for every port that must have it. Using an extended OVERLAPPED structure in the 'usual' way, transfer both the listening and client sockets to the thread that handles the IO completion event.

The handler thread then has access to all per-listener and per-client data as it executes, and so can take the appropriate action. Note carefully that one or both data items may require a lock and/or queue to prevent multiple pool threads from modifying the data due to, say, two clients connecting at the 'same time'.

There is no need for a completion port/pool per listener, one pool is fine.

The handler thread should also issue another AcceptEx() for the next client to connect to.

score 1 · Answer 3 · answered Mar 27 '17 at 04:52

the other answers on this thread are totally legit but I believe I can add a bit to it.

Let us discuss what a completion port IS, the way I understand it.

A completion port is an event sink. That is, an event occurs, a completion port gets notified.

Now, there is of course a large taxonomy of events. Completion port is well-suited for events of a particular kind: IO completions. There is a patent by Cutler and gang written in surprisingly human-readable language, that describes how IOCP integrates with NT kernel.

Recapping: a completion port is an event sink well-suited for receiving notifications of IO completions.

Now let's go back to OP's question:

How do I setup IOCP sockets for multiple listeners (on different ports)?

Your event is a completion of a a previously issued AcceptEx() call. It typically is a connect() call on a client side but also could be someone disabling your NIC.

you first decide if you want the connects on the same port to be processed concurrently or serially. Also you need to decide whether you want to have connects on different ports to be processed concurrently or serially.

I see three potential approaches:

all connection completions are serialized

CreateIOCompletionPort upfront once, BindIOCompletionCallback to it for every listener, one worker thread that calls GetQueuedCompletionStatus as RbMm described

connection completions of same listener (on same port) are serialized, but connection completions on different ports are concurrent

you have to create one completion port per listener and spin up a worker that will GetQueuedCompletionStatus()

all connection completions are concurrent

You can have one completion port but you spin up multiple threads to call GetQueuedCompletionStatus()

You are not necessarily restricted with having thigs as I described above. Completion ports are clever so you can go with one port and multiple waiters, and then waiters acquire per-listener locks for serialization. Completion port machinery in kernel will figure out that you blocked and release other waiters when appropriate.

Recapping: first, you decide how concurrent you want to be. Then you choose how many completion ports you need to have.

Using IOCP with multiple listeners

3 Answers3