5

I am a bit confused as to what actually happens when an IO completion port completes.

I presume that the Win API allows access to an IOCP queue that somehow is able to queue (or stack) a callback reference with a specific handle (let's say a socket). When windows receives an interrupt from the NIC, then it at some point gets to the IOCP queue for the NIC and executes the callbacks on its own (IOCP) thread pool.

My question is, is this thread from the thread pool spawned upon the interrupt being received, or is it in fact spawned when the call to the Win API is made, effectively having the thread in a wait state until it is then woken by the IOCP queue?

EDIT:
I found this: http://msmvps.com/blogs/luisabreu/archive/2009/06/04/multithreading-i-o-and-the-thread-pool.aspx where is states: "Whenever that operation completes, it will queue a packet on that I/O completion port. The port will then proceed and use one of the thread pool’s thread to run the callback you’ve specified."

kasperhj
  • 10,052
  • 21
  • 63
  • 106
  • Just for clarification, this is a question about how overlapped IO manages the reception of event notifications *prior* to bringing it to the IOCP user-managed thread pool, correct ? – WhozCraig Nov 01 '12 at 15:24
  • No. It's a question about whether the threads created by the IOCP are created before there is some IO to the port (in which case they are blocked), or the threads are created at the time the port has some IO to report. – kasperhj Nov 01 '12 at 15:28
  • It isn't very clear what you are talking about. Interrupts are handled by the driver. It matters whether your code uses GetQueuedCompletionStatus() or the later CreateThreadPoolIo(). In the first case you create the tp thread. In the last case Windows creates it when the overlapped I/O completes. – Hans Passant Nov 01 '12 at 16:20
  • @HansPassant That was what I was trying to understand as well. Clearly (to me anyway) he isn't talking about the user-managed thread pool (all those threads calling GetQueueCompletionStatus() and enrolling in the IOCP wait-state). I'm pretty sure its the latter in your description, but you're correct, clarity isnt exactly abundant in this question – WhozCraig Nov 01 '12 at 16:54
  • 4
    I/O completion ports do not create threads. *You* create threads and subscribe them to the IOCP when you call GetQueueCompletionStatus. – Raymond Chen Nov 01 '12 at 19:14
  • Actually, I am only indirectly in contact with the IOCP since I am using C# asynchronous patterns. Probably this is why my question is not very clear -- my knowledge on the subject is not that great. I will elaborate tomorrow. – kasperhj Nov 01 '12 at 20:32
  • @RaymondChen So my threads are blocked until the IOCP unblocks them? – kasperhj Nov 01 '12 at 20:56
  • 2
    @lejon: If you call `GetQueuedCompletionStatus()` with its `dwMillisecond` parameter set to a non-zero value, then yes, the calling thread will be blocked until an IOCP completes or the timeout elapses, whichever occurs first. – Remy Lebeau Nov 01 '12 at 21:25

1 Answers1

7

It's probably easier to think of an I/O completion port simply as a thread safe queue that the operating system places the results of overlapped operations into for you when they have completed.

You create the IOCP, you then create some threads and these threads call a function to remove items from this queue. Generally this is GetQueuedCompletionStatus(). This function essentially blocks your thread until there's something in the IOCP (queue) and then allows your thread to retrieve that something and run.

You associate file handles and sockets with the IOCP and this simply means that once associated their overlapped completions will be placed in the IOCP (queue) for you.

It's more complex than that, but that's the way you should be thinking.

Len Holgate
  • 21,282
  • 4
  • 45
  • 92
  • Just to clarify. In my application, I create my _own_ thread pool, and pass a thread pool thread to the IOCP which is then blocked by the IOCP until something completes? It's difficult to understand because there seems to different "opinions" on how this actually works, e.g. see my edit. – kasperhj Nov 02 '12 at 09:19
  • The threads in your pool will block on the IOCP waiting for work, everything else is the same as stated in my answer. How the pool decides to change the number of threads that it holds is not in any way related to the IOCP. It may be able to monitor when its threads have called GetQueuedCompletionStatus() and if none currently have and some timeout expires then it may create more threads and they will then block on the IOCP waiting for work. I don't know the implementation of the thread pool so I can't say any more. – Len Holgate Nov 02 '12 at 15:58
  • Suppose I call IOCP on the UI thread, it would block until I get my IOCP packet? Do you have a reference to the IOCP internals? – kasperhj Nov 05 '12 at 07:06
  • By "call IOCP" I assume you mean make a call to GetQueuedCompletionStatus() or equivalent? If so then yes it will block until a completion packet is available. No I don't have any references to IOCP internals themselves. I've just read the MSDN documentation and Richter's books and other stuff. – Len Holgate Nov 05 '12 at 08:56
  • I see. It just seems strange that this fact is not clearly described anywhere (that I have been able to find). Just this morning, I was watching slides from a talk by Mads Torgersen, and in the slide the following words appear: "A few seconds later, and the web services has delivered its answer to the IO Completion Port thread!" – kasperhj Nov 05 '12 at 17:23
  • IMHO it doesn't really need more documentation. The docs for IOCPs in general are good, Richter's books are very thorough and assuming you have read and understood how the native APIs work and the implications of using them then it's fairly easy to apply that knowledge to this situation and reason about what's going on (which is a useful skill to acquire). You could, if you wanted, then write some simple test programs to further understand what actually happens. – Len Holgate Nov 05 '12 at 21:02
  • I found the following in the msdn doc on .NET: _"Send/receive operations on a socket can be performed asynchronously with an IOCompletionPort, so when our request is done, the callback function SendCallback is executed on a CompletionPort thread."_, do you have any idea why they write that? – kasperhj Nov 06 '12 at 19:18
  • Do you have a reference to where you obtained that quote, I'd like a little more context... – Len Holgate Nov 06 '12 at 19:57
  • I just realized, that the CLR thread pool implementation may have a separate pool for IOCP, and this is what _actually_ gets blocked. – kasperhj Nov 08 '12 at 08:25