Stack Overflow
Stack Overflow

Questions tagged [python-multithreading]

python-multithreading refers to how to divide work into multiple streams of execution in Python.

python-multithreading refers to how to divide work into multiple streams of execution within a single process in Python. Usually this refers to the threading module. It could also refer to concurrent.futures.ThreadPoolExecutor or the thread/_thread module.

More information:

3995 questions
33
votes
5 answers

ThreadPoolExecutor: how to limit the queue maxsize?

I am using ThreadPoolExecutor class from the concurrent.futures package def some_func(arg): # does some heavy lifting # outputs some results from concurrent.futures import ThreadPoolExecutor with ThreadPoolExecutor(max_workers=1) as…
Bob
  • 5,809
  • 5
  • 36
  • 53
32
votes
2 answers

What happened to thread.start_new_thread in python 3

I liked the ability to turn a function into a thread without the unnecessary line to define a class. I know about _thread, however it appears that you are not supposed to use _thread. Is there a good-practice equivalent of thread.start_new_thread…
lemiant
  • 4,205
  • 4
  • 31
  • 38
32
votes
4 answers

python asyncio, how to create and cancel tasks from another thread

I have a python multi-threaded application. I want to run an asyncio loop in a thread and post calbacks and coroutines to it from another thread. Should be easy but I cannot get my head around the asyncio stuff. I came up to the following solution…
Olivier RD
  • 679
  • 1
  • 5
  • 11
31
votes
2 answers

Matplotlib: simultaneous plotting in multiple threads

I am trying to do some plotting in parallel to finish large batch jobs quicker. To this end, I start a thread for each plot I plan on making. I had hoped that each thread would finish its plotting and close itself (as I understand it, Python closes…
Boris
  • 707
  • 1
  • 7
  • 9
29
votes
6 answers

How to pause and resume a thread using the threading module?

I have a long process that I've scheduled to run in a thread, because otherwise it will freeze the UI in my wxpython application. I'm using: threading.Thread(target=myLongProcess).start() to start the thread and it works, but I don't know how to…
jimbo
  • 369
  • 1
  • 3
  • 7
28
votes
1 answer

Python threads difference for 3.10 and others

For some, simple thread related code, i.e: import threading a = 0 threads = [] def x(): global a for i in range(1_000_000): a += 1 for _ in range(10): thread = threading.Thread(target=x) threads.append(thread) …
Zada Zorg
  • 2,778
  • 1
  • 21
  • 25
28
votes
1 answer

TensorFlow/Keras multi-threaded model fitting

I'm attempting to train multiple keras models with different parameter values using multiple threads (and the tensorflow backend). I've seen a few examples of using the same model within multiple threads, but in this particular case, I run into…
bnaul
  • 17,288
  • 4
  • 32
  • 30
27
votes
3 answers

python logging performance comparison and options

I am researching high performance logging in Python and so far have been disappointed by the performance of the python standard logging module - but there seem to be no alternatives. Below is a piece of code to performance test 4 different ways of…
Sid
  • 7,511
  • 2
  • 28
  • 41
26
votes
2 answers

Python threading error - must be an iterable, not int

I'm trying to calculate rolling r-squared of regression among first column and other columns in a dataframe (first column and second, first column and third etc.) But when I try threading, it kept telling me the error that TypeError:…
Catherine Zhang
  • 315
  • 1
  • 5
  • 8
26
votes
1 answer

Nesting concurrent.futures.ThreadPoolExecutor

I have a program where I am currently using a concurrent.futures.ThreadPoolExecutor to run multiple tasks concurrently. These tasks are typically I/O bound, involving access to local databases and remote REST APIs. However, these tasks could…
Andrew McLean
  • 383
  • 4
  • 11
26
votes
1 answer

Python Celery versus Threading Library for running async requests

I am running a python method that parses a lot of data. Since it is time intensive, I would like to run it asynchronously on a separate thread so the user can still access the website/UI. Do threads using the "from threading import thread" module…
ninajay
  • 527
  • 1
  • 5
  • 10
26
votes
3 answers

Use of threading.Thread.join()

I am new to multithreading in python and trying to learn multithreading using threading module. I have made a very simple program of multi threading and i am having trouble understanding the threading.Thread.join method. Here is the source code of…
Pratik Singhal
  • 6,283
  • 10
  • 55
  • 97
25
votes
2 answers

What is the difference between .Semaphore() and .BoundedSemaphore()?

I know that threading.Lock() is equal to threading.Semaphore(1). Is also threading.Lock() equal to threading.BoundedSemaphore(1) ? And newly I saw threading.BoundedSemaphore(), what is the difference between them? For example in the following code…
Benyamin Jafari
  • 27,880
  • 26
  • 135
  • 150
25
votes
3 answers

Why coroutines cannot be used with run_in_executor?

I want to run a service that requests urls using coroutines and multithread. However I cannot pass coroutines to the workers in the executor. See the code below for a minimal example of this issue: import time import asyncio import…
zeh
  • 1,197
  • 2
  • 14
  • 29
25
votes
1 answer

thread starts running before calling Thread.start

t1=threading.Thread(target=self.read()) print("something") t2=threading.Thread(target=self.runChecks(), args=(self,)) self.read runs indefinitely, so the program won't ever reach the print line. How is this possible without calling t1.start()?…
Tyler Durden
  • 2,031
  • 4
  • 20
  • 25
1 2
3
…
99 100