IOCP loop termination may cause memory leaks? How to close IOCP loop gracefully

Question

I have the classic IOCP callback that dequeues i/o pending requests, process them, and deallocate them, in this way:

struct MyIoRequest { OVERLAPPED o; /* ... other params ... */ };
bool is_iocp_active = true;

DWORD WINAPI WorkerProc(LPVOID lpParam)
{
    ULONG_PTR dwKey;
    DWORD dwTrans;
    LPOVERLAPPED io_req;
    while(is_iocp_active)
    {
        GetQueuedCompletionStatus((HANDLE)lpParam, &dwTrans, &dwKey, (LPOVERLAPPED*)&io_req, WSA_INFINITE); 
       // NOTE, i could use GetQueuedCompletionStatusEx() here ^ and set it in the 
       // alertable state TRUE, so i can wake up the thread with an ACP request from another thread!

        printf("dequeued an i/o request\n");
        // [ process i/o request ]
        ...

        // [ destroy request ]
        destroy_request(io_req);
    }
    // [ clean up some stuff ] 
    return 0;
}

Then, in the code I will have somewhere:

MyIoRequest * io_req = allocate_request(...params...);
ReadFile(..., (OVERLAPPED*)io_req);

and this just works perfectly.

Now my question is: What about I want to immediately close the IOCP queue without causing leaks? (e.g. application must exit) I mean: if i set is_iocp_active to 'false', the next time GetQueuedCompletionStatus() will dequeue a new i/o request, that will be the last i/o request: it will return, causing thread to exit and when a thread exits all of its pending i/o requests are simply canceled by the system, according to MSDN.

But the structures of type 'MyIoRequest' that I have instanced when calling ReadFile() won't be destroyed at all: the system has canceled pending i/o request, but I have to manually destroy those structures I have created, or I will leak all pending i/o requests when I stop the loop!

So, how I could do this? Am I wrong to stop the IOCP loop with just setting that variable to false? Note that is would happen even if i use APC requests to stop an alertable thread.

The solution that come to my mind is to add every 'MyIoRequest' structures to a queue/list, and then dequeue them when GetQueuedCompletionStatusEx returns, but shouldn't that make some bottleneck, since the enqueue/dequeue process of such MyIoRequest structures must be interlocked? Maybe I've misunderstood how to use the IOCP loop. Can someone bring some light on this topic?

Beware of premature stoptimization. – Martin James Mar 21 '13 at 13:00 — Martin James, Mar 21 '13 at 13:00

score 5 · Accepted Answer · answered Mar 22 '13 at 08:37

5

The way I normally shut down an IOCP thread is to post my own 'shut down now please' completion. That way you can cleanly shut down and process all of the pending completions and then shut the threads down.

The way to do this is to call PostQueuedCompletionStatus() with 0 for num bytes, completion key and pOverlapped. This will mean that the completion key is a unique value (you wont have a valid file or socket with a zero handle/completion key).

Step one is to close the sources of completions, so close or abort your socket connections, close files, etc. Once all of those are closed you can't be generating any more completion packets so you then post your special '0' completion; post one for each thread you have servicing your IOCP. Once the thread gets a '0' completion key it exits.

answered Mar 22 '13 at 08:37

Len Holgate

21,282
4
45
92

The problem is in detemining whether the CancelIOEx operation is complete and all outstanding 'calcelled' completion messages have been processed. The outstanding WSAblah would issue a flood of completion messages and it may be necessary to use a server-wide atomic refCount to determine that all those completions have been processed before issuing the 'null message' poison-pills. One possible way out would be for the pool threads to requeue a received pill before terminating so that the other threads get it. That means only one pill is needed for any number of pool threads. – Martin James Mar 22 '13 at 13:48
There's no need to use CancelIOEx at all, IMHO. You just let the operations complete with errors when the underlying sockets and handles are closed. You don't have to wait for processing to be complete before sending the 0 completions (one per thread), just close the sockets/handles first, then issue the 0 byte completions. The closure will trigger the completion of the outstanding operations and the threads will process them all and then shut down. I've been doing it this way for over 10 years with a very diverse set of server designs for various clients and, it just works. – Len Holgate Mar 22 '13 at 14:44
Hi Len, humbled to see you here. Well, this is exactly what I do, I send a packet with a 'exit' operation to all threads. The problem here is that I have other completion packets queued _after_ the exit packet, so the thread exits, nobody dequeue that other packets and it will leak. So in your opinion the correct behavior for IOCP is denying the queueing of packets _after_ the 'exit' packets are sent? Do you think I have to synchronize it better, so after the 'exit' packets no other completion requests are sent to the completion port? – Marco Pagliaricci Mar 22 '13 at 17:55
The problem of this, is that i'm using the IOCP threadpool also for async tasks, so for example ayncClose(handle); that will call CloseHandle inside on one thread of the threadpool, and in turn will call a callback [e.g. onClosed()]. Now, the problem is that a CloseHandle() of the socket is *queued*, but not executed yet, so in that time the socket can receive data, so other OVERLAPPED structures are instanced! So if I call asyncClose() for all the sockets, and then immediately I send the 'exit' packets to all threads, I could have some leaks! – Marco Pagliaricci Mar 22 '13 at 17:56
A solution would be to wait until _all_ onClosed() are called on _all_ HANDLEs open and _after_ send 'exit' packets, but its difficult: how? An interesting solution I thought to this is to send a 'wake-exit' packet, that simply makes GetQueuedCompletionStatus() to wake up, set a flag that says "IOCP is done" without exiting from the loop [i mean leaving is_iocp_active==true] and start calling GQCS () with its 'dwMilliseconds' parameter set to 0, and if it returns a request simply destroy the request avoiding the leak, and finally when GQCS returns that the queue is empty, the thread return 0; – Marco Pagliaricci Mar 22 '13 at 17:58
What you think about this solution? It apparently works, but I have to test it more. Do you think this would be reliable, or I have to just wait all HANDLE are closed and then send a simple 'exit' packets to all threads? [I'm sorry for the long comments] – Marco Pagliaricci Mar 22 '13 at 18:01
If you are posting your own operations to the IOCP then I simply have a "pool has shutdown" flag which I check before posting my operations. If the flag is set the op goes straight to a simple handler than can clean up the resources but doesn't perform the operation. If you have APIs that post to the IOCP then I suggest you close/cancel those APIs if possible before sending your 0. – Len Holgate Mar 23 '13 at 08:28
I can see how the "shut down when you're empty" that you're doing would work, but I expect there's still a bit of a race condition in there and always will be no matter what delay you use as you suggest that you don't know when things will stop posting to your IOCP. – Len Holgate Mar 23 '13 at 08:28
I see. Yes I have added such flag that blocks posting of operations when pool is shutting down and just deletes them. But first introducing it, I indeed incurred in a race condition: with my method of "shutdown when you're empty" when I called GetQueuedCompletionStatus() with a delay of 0 it _worked_ without problems, but if I called it with a delay of 1 or more (not INFINITE, of course) it deadlocked! The application freezed when closing the threadpool, and sometime it closed 4,6 of the 8 threads I had. It deadlocked in XP and also in Windows7 -- but why? I can't see the race condition here – Marco Pagliaricci Mar 25 '13 at 08:43
Martin, just realised that I DO use CancelIO() very occasionally to cancel OOB reads if one is pending, but that's just due to the peculiarities of my framework. – Len Holgate Mar 25 '13 at 10:58

Martin James · Answer 2 · 2013-03-21T13:06:07.530

-1

If you are terminating the app, and there's no overriding reason to not do so, (eg. close DB connections, interprocess shared memory issues), call ExitProcess(0).

Failing that, call CancelIO() for all socket handles and process all the cancelled completions as they come in.

Try ExitProcess() first!

edited Mar 21 '13 at 13:06

answered Mar 21 '13 at 12:55

Martin James

24,453
3
36
60

Well, with ExitProcess() I'm almost sure that the memory will be free'd, but I've asked that when I just want to stop the loop. Thats because in C++ I have an object that abstracts an IO Completion Port, so, when I call terminate() on that object, I want to stop all of its IO Completion Ports loops. With CancelIO() I cancel the I/O request (e.g. the request will be removed from a list?) but I'm not sure if I can avoid the memory leak, because the io_req object will still be there... – Marco Pagliaricci Mar 21 '13 at 14:26
They will be returned in the flood of completion messages after CancelIO(), no? – Martin James Mar 21 '13 at 14:31
Also - don't need a flag. Post a 'quit now' poison-pill message on the IOCP queue. – Martin James Mar 21 '13 at 14:33
Yeah, I usually post a 'quit now' message on the IOCP queue, but what about the queue is very long? It will take a while, if my application must close IOCP right now, with the flag you can do that immediately. – Marco Pagliaricci Mar 21 '13 at 17:11
Well, yes, depends on how many entries in the completion queue. If there are none, because the server is not busy, your pool threads will not see the flag. – Martin James Mar 21 '13 at 17:24
Ok, I see. I still have some questions about CancelIO(). If I got it, I have to use it from the same thread that created the io request, but since when I read sockets or files in an asyncrhonously way, I perform io operations in random threads from the threadpool, e.g. a Read io request is completed? let's read another piece of socket/file with ReadFile() and a new io request <- and this happen in one random thread that completed the previous request. So, how I can call CancelIO() if I don't know which thread fired the io request? – Marco Pagliaricci Mar 21 '13 at 18:37
And what about I don't have an io request but only another kind of post? (e.g. like your 'quit now' message, but different: 'run this' message that run a my own Runnable object, to let iocp threadpool perform tasks) <- in this case CancelIO() is not applicable, i guess. So what about the IOCP queue has a lot of Runnable pending tasks? With the flag set to false, I can obtain the thread to exit when the next Runnable is finished, but in such case: what about all other pending tasks, and instanced structures? Will they leak? – Marco Pagliaricci Mar 21 '13 at 18:41
@MarcoPagliaricci CancelIOEx? – Martin James Mar 22 '13 at 13:17
As for your other issue of determining when all outstanding IO/etc. requests have been handled, so as to prevent leaks, I have to think some more, (not done this sort of thing except on app exit, when leaks did not matter). – Martin James Mar 22 '13 at 13:21
-1, because naively calling `ExitProcess()` in a multi-threaded, networked application can very easily lead to deadlock, see MSDN article on ExitProcess. – Joker_vD Apr 30 '17 at 22:47

IOCP loop termination may cause memory leaks? How to close IOCP loop gracefully

2 Answers2