I'm writing a python 3.6 application which make a heavy use of threads (as a design pattern, a directed graph of threads linked with queue.Queue as edges) and it works fine.
One thread makes use of the sound card with pyaudio and buffer the samples to output (there is at any time as much of samples as required), one thread makes use of sklearn's IncrementalPCA, doing heavy computational work sometimes (high CPU requirements, ~30%, with high memory requirements, ~8Mb).
When the heavy computational work is running in the latter thread, I got a (snd_pcm_recover) underrun occurred from the sound card, just as the first thread isn't doing its buffering well (but it does buffer well, the samples are ready to be sent when the underrun occurs).
There is no mutex nor synchronized code between the two threads, they are two independent pieces of code. Everything works fine when I reduce the computational heavy work by a 2 factor, getting something like ~15% CPU with 4Mb of memory consumed by the thread.
As sklearn's IncrementalPCA spend most time using numpy and as I use python 3.6, I don't think that the problem is related to Python's GIL, but I may be misunderstanding something.
I've think about throttling the heavy computational work of the latter thread, but it seems unlikely to be even possible. I'm running short of ideas, so:
- How can I make the first thread working with the latter thread without underruns ?
