Stack Overflow
Stack Overflow

Questions tagged [concurrent.futures]

concurrent.futures is a Python module which provides a high-level interface for asynchronously executing callables.

The concurrent.futures module aims to provide a simple interface for parallelizing operations in multi-threaded and multi-process Python applications. The module was added to the Python standard library in version 3.2, but a backport is available for Python 2.5+.

929 questions
0
votes
1 answer

Check if parallel execution is happening in Python

I have the following code, which uses selenium to open 10 google search pages and do a search all 10 searches at the same time. I think it runs correctly (in parallel), but I'm not sure, because it actually seems to open 5 instances of the browser…
Ke.
  • 2,484
  • 8
  • 40
  • 78
0
votes
1 answer

Can we replace urlopen in this code with requests library?

Can we replace urlopen library in this example for concurrent requests with the requests library in python 2.7? import concurrent.futures import urllib.request URLS = ['http://www.foxnews.com/', 'http://www.cnn.com/', …
Павел Иванов
  • 1,863
  • 5
  • 28
  • 51
0
votes
1 answer

How are values transferred across threads in scala futures

How is num being accessed by the new thread. Future will execute on the new thread. how is the value of num that is in the stack frame of main thread accessible in the new thread? Where is it stored? object TestFutureActor extends App { var num =…
Arpan Patro
  • 153
  • 2
  • 8
0
votes
1 answer

assertRaises not working with Futures

I changed a call to bar which returned synchronously to returning a Future. When the method of bar actually gets executed it raises an ValueError. The unit tests actually ensured that. Now changing self.assertRaises(ValueError,…
abergmeier
  • 13,224
  • 13
  • 64
  • 120
0
votes
1 answer

How does the FutureTask in collection works internally?

I am trying to understand the working of FutureTask in Collections. From what I read I understand you can create a thread pool using ExecutorService. Later you can wrap Runnable or Callable in FutureTask and execute it. After that you can use the…
krs8888
  • 1,239
  • 4
  • 19
  • 26
0
votes
1 answer

concurrent.futures.ThreadPoolExecutor max_workers can't be 0

If I spin up a ThreadPoolExecutor(max_workers=0) it works with Python3.4 and Python2.7 but raises an error with Python3.5 and Python3.6. I'm trying to create a ThreadPoolExecutor where I want to ensure that no task gets added to the threadpool.…
Sid Shukla
  • 990
  • 1
  • 8
  • 33
0
votes
2 answers

concurrent.futures not parallelizing write

I have a list dataframe_chunk which contains chunks of a very large pandas dataframe.I would like to write every single chunk into a different csv, and to do so in parallel. However, I see the files being written sequentially and I'm not sure why…
elelias
  • 4,552
  • 5
  • 30
  • 45
0
votes
1 answer

Insert data into Postgresql from Python

≈105 seconds per 1 million rows to insert into Postgresql local database on table with 2 indexes and 4 columns it is slow or fast ? Python Code: import os import pandas as pd from concurrent.futures import ThreadPoolExecutor, as_completed from…
Kostiantyn Palianychka
  • 584
  • 1
  • 7
  • 22
0
votes
0 answers

Scala how to recover exception in a for future

Here is my question! When I use several future tasks in a for expression, I'm confused with the result of exception recover. object ForTest { def ft1(n: Int): Future[Int] = Future(n*2) def ft2(n: Int): Future[Int] = Future(60/n) def…
Shihan Wang
  • 11
0
votes
1 answer

With threading I pick up 9 'hits', without threading I pick up 214. What's going on?

The basic purpose of my script is to filter through a range of numbers (say, 5000),the numbers that are valid are saved to a list called hit_list. The real range I'm looping through is much bigger than 5000, so I need concurrency to make the time…
SeánMcK
  • 392
  • 3
  • 17
0
votes
1 answer

Asynchronous processing of streaming HTTP with requests / requests_futures

I understand how to stream data over HTTP with requests: import requests r = requests.get(url, stream=True) for line in r.iter_lines(): print(line) I understand how to process a request asynchronously: from requests_futures.sessions import…
max
  • 49,282
  • 56
  • 208
  • 355
0
votes
0 answers

Time based aggregator in Java

The problem Required is a service, initialized with a pre-set time span (for example 3 seconds), that receives work (objects) over time and each time 3 seconds have passed with no new work, it releases all the accumulated work as a collection. Each…
D. L.
  • 481
  • 4
  • 5
0
votes
1 answer

Pickling error on concurrent.Futures.PoolProcessExecutor - How to Lock?

I have been messing around with Multiprocessing on and off for months now trying to figure out an elegant repeatable solution to my issue of wanting multiple processes to write to the same file without messing each other up. I have used the…
ccdpowell
  • 629
  • 5
  • 14
  • 22
0
votes
2 answers

python concurrent process -electrical engineering usage

I am an electrical engineer trying to do multiprocessing in python2.7. I have two oscilloscopes which need to run the same tests on 2 different signals. Right now, I have a code which does it in sequence and takes long time. I want to make the…
Yash
  • 1
  • 2
0
votes
1 answer

using futures.concurrent with a class

I have a main that looks like the following: import gather_filings_per_filer import os from concurrent import futures def put_last_i(arg): with open(os.path.dirname(os.path.abspath(__file__)) + '/last_i.txt','w') as f: …
jason m
  • 6,519
  • 20
  • 69
  • 122
1 2 3
…
61 62