Questions tagged [multiprocessing-manager]
88 questions
0
votes
1 answer
Python sharing multiprocessing.Manager.list with child processes
My question is so simple. I want to share a multiprocessing.Manager().dict() with child processes. But shared dict will be initialized AFTER starting childs. Here is the example code:
import multiprocessing
import time
class Singleton:
…
Syrenthia
- 145
- 2
- 11
0
votes
0 answers
pass object of selenium from a subprocess to another subprocess
I try to use manager and queue to share the object of selenium in two subprocess, but it shows me the error "AttributeError: Can't pickle local object '_createenviron..encodekey'"
from selenium.webdriver import Chrome
from multiprocessing import…
Nicholas
- 1
- 1
0
votes
0 answers
Why python shows "multiprocessing.managers.RemoteError"?
I'm trying to use selenium with multiprocessing manager to grabbing courses, one subprocess to find if anyone quit the course and the other one refreshing the adding courses' page to prevent connection timed out. But it shows me the wrong message…
Nicholas
- 1
- 1
0
votes
0 answers
Why am I getting an error from the following usage of multiprocessing.Manager().Queue()
I wrote a script and ran it on the darwin system. In order to maximize the efficiency of my script (bypassing the GIL), I wrote the code with the following structure:
import multiprocessing as mp
import threading
import queue
p_num = 2
t_num =…
m276
- 1
- 1
0
votes
0 answers
Parallelize prediction from multiple keras models
As stated in the title I'm trying to parallelize the prediction of multiple (sequential) keras models.
The challenge is that I have ~200 small models (each for a different signal, each trained individually - so combining them to a larger model is…
Viktor Katzy
- 162
- 10
0
votes
0 answers
Bug or error when attempting to append process to list/array and return
When attempting to append process id to list inside processing_task the list is changed but when attempting to return it from function it is unchanged for some reason I don't understand what the problem is...
Goal is :
1.Pass all_process list to…
Ggjustice V
- 75
- 8
0
votes
0 answers
Is multiprocessing.Manager.Queue() thread and process safe?
I am fairly new to multiprocessing in python. I am trying to use Queues to pass messages between processes. As I read through the documentation it says multiprocessing.Queue() is thread and process safe. But it doesn't say whether…
0
votes
0 answers
What makes Python Multiprocessing raise different errors when sharing objects between processes?
Context: I want to create attributes of an object class in parallel by distributing them in the available cores. This question was answered in this post here by using the python Multiprocessing Pool.
The MRE for my task is the following using Pyomo…
Yuri Santos
- 55
- 7
0
votes
1 answer
Accessing multiprocessing manager dict from webserver
I've been trying to share a multiprocessing manager dict with a web server so far with little success. The dict contains information but somehow it is not propagated to the running web server. The goal ultimately is to be able to intermittently poll…
Dr Acme Isme
- 55
- 4
0
votes
1 answer
Unreliable multiprocessing Manager
I was having a problem with some more complex code but have reduced it to something that reproduces the issue:
from multiprocessing import Manager
from concurrent.futures import ProcessPoolExecutor
from itertools import repeat
def process(v):
…
DarkKnight
- 19,739
- 3
- 6
- 22
0
votes
1 answer
Python multiprocessing can't use functions from other module
Update: it's working after updating my Spyder to 5.0.5. Thanks everyone!
I am trying to speed up a loop using multiprocessing. The code below aims to generate 10000 random vectors.
My idea is to split the task into 5 processes and store it in…
Meinung
- 133
- 6
0
votes
1 answer
Sharing a list between processes with multiprocessing.Manager
1st
'''
import multiprocessing as mp
def func( list):
x = 5
list = list*5
print('list_in_sub: ', list)
if __name__ == "__main__":
list = mp.Manager().list(range(2))
p1 = mp.Process(target=func, args=( list,))
p1.start()
…
xibuzuoluo
- 1
- 2
0
votes
1 answer
How to share complex objects in multiprocessing manager
I know this question is being asked in many threads but so far I couldn't find proper solution.
Below is my code:
Class A:
def init(self):
self.b = B()
Class B:
def init(self):
....
In the main function
d = mp.Manager().dict()
p…
python_interest
- 874
- 1
- 9
- 27
0
votes
1 answer
How do read and writes work with a manager in Python?
Sorry if this is a stupid question, but I'm having trouble understanding how managers work in python.
Let's say I have a manager that contains a dictionary to be shared across all processes. I want to have just one process writing to the dictionary…
Justin Furuness
- 685
- 8
- 21
0
votes
1 answer
Python multiprocessing manger.list() not being pass to Matplotlib.animation correctly
I have this 2 processes with a list created with manager.list() being shared between them, one is called DATA() and it is "generating" data and appending to the list and the other is plotting that data using Matplotlib animation FuncAnimation.
The…
fagner nunes
- 37
- 6