1

I'm currently using asyncio in Python 3.7 and write a TCP server using the asyncio.start_server() function refer to this example: https://docs.python.org/3/library/asyncio-stream.html

Also try asyncio.ProactorEventLoop that uses “I/O Completion Ports” (IOCP)

According to this Microsoft official doc https://learn.microsoft.com/en-ca/windows/win32/fileio/i-o-completion-ports, it mention that using I/O completion ports with a pre-allocated thread pool but I cannot find where can allocate the number of thread

Where can I allocate the number of thread in thread pool? Can anyone please help me here? Thanks a lot!

Henry Liang
  • 11
  • 1
  • Why do you care about the number of threads in the OS-provided thread pool? I suspect you won't be able to access those threads from Python code. – user4815162342 Jun 14 '20 at 08:53
  • Because I want model threads to handle more request. such as handle thousands of request within one second As c++ can set NumberOfConcurrentThreads parameter in CreateIoCompletionPort function – Henry Liang Jun 16 '20 at 04:45
  • Are you aware that asyncio is single threaded? Bumping the number of threads uses by completion ports won't change that. – user4815162342 Jun 16 '20 at 05:39
  • I think asyncio.start_server() only use single thread but asyncio library use coroutine that we have nearly all the advantages of multi-threading, without actually using multiple threads – Henry Liang Jun 16 '20 at 05:49
  • That's correct, and is also the reason why the number of concurrent threads used by completion ports is unlikely to affect performance. I strongly suspect these threads need to be used by user code to actually make a difference, and Python+asyncio will execute everything in the thread that runs the event loop. You probably need to find some other way to increase the performance of your program. – user4815162342 Jun 16 '20 at 06:14
  • Do you mean allocate more threads cannot handle thousands of request within one second? If so, how can I do this by using asyncio ? – Henry Liang Jun 16 '20 at 06:14
  • I mean allocating any number of threads won't help with a single-threaded framework such as asyncio. I can't help you with your problem because you didn't describe what it is you are attempting to solve. Is asyncio too slow? How are you using it, can you show a minimal example that demonstrates the issue? – user4815162342 Jun 16 '20 at 06:20

1 Answers1

2

at first general info about I/O completion ports (iocp) and thread pool(s). we have 2 options here:

create all by self:

create iocp by self via CreateIoCompletionPort (or NtCreateIoCompletion).

by self create threads, which will be call GetQueuedCompletionStatus (or NtRemoveIoCompletion).

every file you need bind to your iocp again by self via NtSetInformationFile with FileCompletionInformation and FILE_COMPLETION_INFORMATION or via CreateIoCompletionPort (this win32 api combine functional of NtCreateIoCompletion and NtSetInformationFile).

use system iocp(s) and thread pool(s).

system (ntdll.dll) create default thread pool (now it named TppPoolpGlobalPool) when process startup. you have week control for this pool. you can not got it direct pointer PTP_POOL. exist undocumented TpSetDefaultPoolMaxThreads (for set the maximum number of threads in this pool) but no for minimum.

if want - you can create additional thread pools via CreateThreadpool function.

After creating the new thread pool, you can (but not should!) call SetThreadpoolThreadMaximum to specify the maximum number of threads that the pool can allocate and SetThreadpoolThreadMinimum to specify the minimum number of threads available in the pool.

The thread pool maintains an I/O completion port. the iocp created inside call CreateThreadpool - we have no direct access to it.

so initially in process exist one global/default thread pool (TppPoolpGlobalPool) and iocp (windows 10 for parallel loader create else one thread pool LdrpThreadPool but this of course only for internal use - while DDLs loading)

finally you bind self files to iocp by call CreateThreadpoolIo

note that msdn documentation is wrong here -

Creates a new I/O completion object.

really CreateThreadpoolIo function not create new I/O completion object - it created only inside call CreateThreadpool. this api bind file (not handle but file!) to I/O completion object which is associated to pool. to which pool ? look for last parameter - optional pointer to the TP_CALLBACK_ENVIRON.

you can specify a thread pool in next way - allocate callback environment, call InitializeThreadpoolEnvironment for it and then SetThreadpoolCallbackPool.

If you do not specify a thread pool, the global thread pool will be used in call CreateThreadpoolIo - so file will be bind to default/global process iocp

and you not need by self call GetQueuedCompletionStatus (or NtRemoveIoCompletion) in this case - system do this for you from pool. and then call your IoCompletionCallback callback function, which you pass to system inside CreateThreadpoolIo call

we can also use system global thread pool and iocp via BindIoCompletionCallback ( or RtlSetIoCompletionCallback) - it associates the I/O completion port owned by the global (TppPoolpGlobalPool) thread pool with the specified file handle. this is old api and variant of case 2. here we can not use custom poll - only process global.

now let back to concrete Python code. which case it use ? are it create iocp and thread pool by self ? or it use system thread pool ? if use system - use it global or custom thread pool allocated by CreateThreadpool ? if you dont know this - nothing can be done here. and even if know.. or this library have special api/interface (or how this in python called) for control this (in case self or custom pool used) or you only can use it as is. and really hard decide how many threads you really need in pool

RbMm
  • 31,280
  • 3
  • 35
  • 56