Stack Overflow
Stack Overflow

Questions tagged [concurrent.futures]

concurrent.futures is a Python module which provides a high-level interface for asynchronously executing callables.

The concurrent.futures module aims to provide a simple interface for parallelizing operations in multi-threaded and multi-process Python applications. The module was added to the Python standard library in version 3.2, but a backport is available for Python 2.5+.

929 questions
0
votes
1 answer

Python concurrency with imported modules

Why does importing local modules cause Concurrent.futures.ProcessPoolExecutor to throw a BrokenProcessPool exception? When I import a local module, I get a BrokenProcessPool exception when I run the file. I have tried commenting out everything in…
David
  • 300
  • 1
  • 14
0
votes
0 answers

Is there anyway to put python futures into a fifo?

I'm trying to put together a parallel processing python script. It uses futures to perform a heavy duty processing task, while the main thread builds tasks to for them to execute, and places them into a multiprocessing.Queue. There's another process…
HSchmale
  • 1,838
  • 2
  • 21
  • 48
0
votes
1 answer

Python Z3 and concurrent.futures

I want to solve a set of contained problems in parallel, after which addition information is added to solve a new problem. Below is an example of the structure of the program used to solve the problem: from z3 import * import concurrent.futures #…
Johan
  • 575
  • 5
  • 21
0
votes
1 answer

Concurrent futures webscraping

I am currently trying to develop a fast webscraping function so I can scrape a large list of files. This is the code I have currently: import time import requests from bs4 import BeautifulSoup from concurrent.futures import ProcessPoolExecutor,…
Bashir
  • 61
  • 8
0
votes
2 answers

How to buffer logs from multithreaded function call so that they can be logged in order the functions finish?

the problem I'm trying to use the concurrent.futures library to run a function on a list of "things". The code looks something like this. import concurrent.futures import logging logger = logging.getLogger(__name__) def process_thing(thing,…
leafmeal
  • 1,824
  • 15
  • 15
0
votes
1 answer

Extract value from AsyncEither - ScalaZ

I am new in ScalaZ and in Scala in general so I faced the issue below. I wrote some unit tests for my application and what I need is to extract the value of the actual response. May you have any hint on how I can do it? Expected…
pik4
  • 1,283
  • 3
  • 21
  • 56
0
votes
2 answers

Python process not cleaned for reuse

Processes not cleaned for reuse Hi there, I stumbled upon a problem with ProcessPoolExecutor, where processes access data, they should not be able to. Let me explain: I have a situation similar to the below example: I got several runs to start with…
Dave J
  • 475
  • 9
  • 18
0
votes
1 answer

Why is there an asyncio.Future error raised by a concurrent.futures.Future?

I have something equivalent to the following code snippet: import asyncio futures = [] loop = asyncio.get_event_loop() for coroutine in coroutines: futures.append(asyncio.run_coroutine_threadsafe(coroutine, loop)) for future in futures: …
insysion
  • 301
  • 2
  • 9
0
votes
1 answer

Python current.futures import libraries multiple times (execute code in top scope multiple times)

for the following script (python 3.6, windows anaconda), I noticed that the libraries are imported as many as the number of the processors were invoked. And print('Hello') are also executed multiple same amount of times. I thought the processors…
casbby
  • 896
  • 7
  • 20
0
votes
1 answer

Python concurrent.futures performance difference with little change

i got the following code: from concurrent.futures import ThreadPoolExecutor, wait import datetime def calculate_mass(mass): for _ in range(10000): gravity = 9.8 weight = mass * gravity def perform_calculations(): with…
jhonatan teixeira
  • 760
  • 1
  • 6
  • 8
0
votes
0 answers

Future cancellation in java based on names or an ID

I am scheduling a future for execution while adding an item in a cart.Now I want to cancel all the previous instances of Future callbacks which were scheduled when an item is updated in the cart. I have scheduled my future like for a single item add…
Ayush Srivastava
  • 444
  • 1
  • 4
  • 13
0
votes
1 answer

Python ThreadPoolExecutor not executing proper

Im using concurrent.futures library to do a for-loop with multithreading. It needs to do the for loop every time with all 5 parameters. Now i have reached the point that my do_something_parallel-function only prints "test1" and nothing more. The…
KnakworstKoning
  • 361
  • 3
  • 6
  • 18
0
votes
0 answers

Concurrent parallel execution

I am trying to execute several python code at the same time in different folder using concurrent futures. The code run fine but when i check the subfolders I don't see any output so I am assuming the python code that I am calling were never…
Angel_M
  • 97
  • 9
0
votes
0 answers

Never entering multithread method in Python

I'm trying to execute the do_shit_parallel function with multithreading. def do_shit_parallel(par1,par2,par3,par4,par5): print("function entered") with ThreadPoolExecutor(max_workers=4) as executor: for LinkedConnector in…
KnakworstKoning
  • 361
  • 3
  • 6
  • 18
0
votes
1 answer

Building large xml file with python 2

I'm trying to get the best performance for building a large XML file in Python 2/Django. The final XML file is ~500mb. The 1st approach used was with lxml, but it took over 3.5 hours. I tested with xml.sax (XMLGenerator) and took about the same…
xampione
  • 71
  • 1
  • 6
1 2 3
…
61 62