Stack Overflow
Stack Overflow

Questions tagged [concurrent.futures]

concurrent.futures is a Python module which provides a high-level interface for asynchronously executing callables.

The concurrent.futures module aims to provide a simple interface for parallelizing operations in multi-threaded and multi-process Python applications. The module was added to the Python standard library in version 3.2, but a backport is available for Python 2.5+.

929 questions
4
votes
2 answers

Does a thread really start after std::future::get() is called?

I created a for loop to define a std::vector of std::future to execute my function vector identify and another loop to get the results by calling std::future::get() as follows: for (int i = 0; i < NUM_THREADS; ++i) { …
Duc Nguyen
  • 721
  • 1
  • 6
  • 16
4
votes
1 answer

Understanding concurrent.futures.Executor.map()

I'm trying to parallelize some Python code using processes and concurrent.futures. It looks like I can execute a function multiple times in parrallel either by submitting calls and then calling Future.result() on the futures, or by using…
planetp
  • 14,248
  • 20
  • 86
  • 160
4
votes
2 answers

Should API return CompletionStage or CompletableFuture

When building an API it is a good practice to code to an interface so it seems like returning CompletionStage seems like a best approach. However I realized that I happen to be always calling .toCompletableFuture after getting the CompletionStage.…
user_1357
  • 7,766
  • 13
  • 63
  • 106
4
votes
1 answer

Why lambda inside map is not running?

I am trying to learn concurrency and lambdas in java 8. But my code is not entering lambda block inside map. List bookList = new ArrayList(); isbnList .stream() .map(isbn -> (CompletableFuture.supplyAsync( () -> { …
Manvi
  • 1,136
  • 2
  • 18
  • 41
4
votes
0 answers

Python Hanging Threads

I have the following code: final = [] with futures.ThreadPoolExecutor(max_workers=self.number_threads) as executor: _futures = [executor.submit(self.get_attribute, listing, …
pornstache
  • 193
  • 3
  • 9
4
votes
2 answers

Subclassing Future Class

I would like to subclass the Future class of the concurrent Python module. The docs: The Future class encapsulates the asynchronous execution of a callable. Future instances are created by Executor.submit(). The docs of Executor don't explain…
guettli
  • 25,042
  • 81
  • 346
  • 663
4
votes
1 answer

Python 3 concurrent.futures and per-thread initialization

In Python 3, is it possible to use a subclass of Thread in the context of a concurrent.futures.ThreadPoolExecutor, so that they can be individually initialized before processing (presumably many) work items? I'd like to use the convenient…
Pierre D
  • 24,012
  • 7
  • 60
  • 96
4
votes
0 answers

Why use concurrent.futures instead of multiprocessing pools?

I'm having trouble understanding why I would use the concurrent.futures module. The ProcessPoolExecutor seems comparable to the multiprocessing pool for processes, and likewise for the ThreadPoolExecutor vs the multiprocessing.dummy module for…
skrrgwasme
  • 9,358
  • 11
  • 54
  • 84
4
votes
1 answer

Python-get function's return value when called using Concurrent.Futures module as ThreadPool

I have created a ThreadPool using Concurrent.Futures module. I have created threads and everything worked fine. But now i have a function which i want to make it parallel with threads, the problem is i need the return value of the function each…
MayaC
  • 61
  • 1
  • 7
4
votes
1 answer

recursion max error when when using futures.ProcessPoolExecutor but not futures.ThreadPoolExecutor with PRAW wrapper

I am using this code to scrape an API: submissions = get_submissions(1) with futures.ProcessPoolExecutor(max_workers=4) as executor: #or using this: with futures.ThreadPoolExecutor(max_workers=4) as executor: for s in executor.map(map_func,…
sunny
  • 3,853
  • 5
  • 32
  • 62
4
votes
1 answer

Tornado future result after timeout

This may sound a bit weird, but is it possible that Tornado completes executing a future after wrapping it in a timeout? So something like this: try: result = yield gen.with_timeout(time.time() + 1, future) except gen.TimeoutError as e: …
Mehmet Catalbas
  • 345
  • 1
  • 3
  • 7
4
votes
1 answer

Python concurrent.futures using subprocess with a callback

I am executing a FORTRAN exe from Python. The FORTRAN exe takes many minutes to complete; therefore, I need a callback to be fired when the exe finishes. The exe does not return anything back to Python, but in the callback function I will use…
Flash
  • 543
  • 2
  • 7
  • 21
4
votes
1 answer

Python 3 concurrent.futures: How to add back failed futures to ThreadPoolExecutor?

I've a list of urls to download via concurrent.futures's ThreadPoolExecutor, but there may be some timeout urls which I want to re-download them after all first tries are over. I don't know how to do it, here is my try, but failed with endless…
xiang
  • 393
  • 3
  • 14
4
votes
3 answers

Scala Futures: Default error handler for every new created, or mapped exception

Is there any possibility to always create a Future{...} block with an default onFailure handler? (e.g. write the stacktrace to the console)? This handler should also be automatically attached to mapped futures (new futures created by calling map on…
Chris W.
  • 2,266
  • 20
  • 40
3
votes
1 answer

executor.map and non-terating parameters

I am trying to convert my script from using threads to much cooler multiprocessing (with python 3.2 and concurrent.futures, but this piece of code crashes with ThreadPoolExecutor(max_workers=MAX_THREADS) as executor: for result…
mcepl
  • 2,688
  • 1
  • 23
  • 38
1 2 3
…
61 62