Stack Overflow
Stack Overflow

Questions tagged [multiprocessing-manager]

88 questions
0
votes
1 answer

Change the spawned process name for Python multiprocessing manager

I want to change the process name of the spawned multiprocessing's manager SyncManager using setproctitle, I tried extending multiprocessing.managers.SyncManager: class MySyncManager(multiprocessing.managers.SyncManager): def __init__(self): …
Gotenks
  • 367
  • 6
  • 14
0
votes
1 answer

Python IPC Shared Memory between two scripts

I am trying to develop an application for the Raspberry Pi3. In fact 2 applications. Application 1 will be a simple gpio read, count the number of times a particular gpio input goes high and log into file . Application 2 will be to show the GPIO…
suresh
  • 29
  • 9
0
votes
0 answers

The best way to share a class between processes

First of all, I'm pretty new in multiprocessing and I'm here for learning of all of you. I have several files doing something similar to this: SharedClass.py: class simpleClass(): a = 0 b = "" ..... MyProcess.py: import…
Keles
  • 369
  • 1
  • 2
  • 14
0
votes
0 answers

The read speed of Manager.dict() is too slow

I have a huge dict variable which is about 2 Gigabytes. I am doing some scientific calculation on this dict(read only). However, the reading speed of the shared dictionary is much much slower than a regular dictionary even if it can save a lot of…
Serpent_Beginer
  • 1
  • 2
0
votes
1 answer

Python multiprocessing - Using shared variables with a manager between threads?

I am using a Manager from pythons multiprocessing library. I would like to share variables between threads. So I have a function and this function uses one such variable internally. Because of python's multiprocessing rules, I need to put the…
Boldizsar Zopcsak
  • 39
  • 7
0
votes
0 answers

Multiprocessing manager.list using list subclass

I am trying to use my list subclass on calling manager.list. So, I can add my own methods, but it is not working. Is it possible to do this? class mylist(list): def __init__(self, *args): super().__init__(*args) …
zezo
  • 445
  • 4
  • 16
0
votes
0 answers

multiprocessing.Process does not do anything

I am trying to understand parallel processing using multiprocessing in python using the example given in this link https://pymotw.com/2/multiprocessing/basics.html. Here is the code: def worker(i): """worker function""" print('Worker',i) …
Aman Savaria
  • 164
  • 1
  • 1
  • 11
0
votes
0 answers

How to speed up multiprocessing manager?

I created my own manager since I will eventually need to pass classes into different processes. I did this using the following code. class MyManager(BaseManager): pass MyManager.register('modeling', modelingClass) manager =…
V. M.
  • 65
  • 1
  • 5
0
votes
0 answers

Python multiprocessing: distribute tasks to managers asynchroneously

I need to hold instances of python objects in different processes, call methods of these objects and collect the results asynchroneously. My approach was to create a manager for each process and object and then to call the repsective methods via the…
Samufi
  • 2,465
  • 3
  • 19
  • 43
0
votes
1 answer

Python Multiprocessing: How to make Pool and Manager work together?

I am trying to understand how Pool() and Manager() can be combined in Python's multiprocessing library for having shared objects between the individual worker processes. In my example, I would like to have a shared list input_list, which should be…
proedig
  • 156
  • 2
  • 4
0
votes
0 answers

Running lots of processes modifying the same list with Python Multiprocessing results in OSError: Too many open files

I can't get the following code to execute successfully, always ending up with the following error: OSError: [Errno 24] Too many open files with the most recent call on p.start() As you can see I already tried to execute the processes in chunks of…
Bunny
  • 1
0
votes
1 answer

Adding appending a dictionary to a list inside another multiprocessing.managers.DictProxy

I have a dictionary that is a multiprocessing.managers.DictProxy . The dictionary looks like: manager = multiprocessing.Manager() data_set = manager.dict() def write_company(company): try: entity = company["registration"]["name"] …
salarhaj
  • 11
  • 3
0
votes
1 answer

multiprocessing.pool with manager and async methods

I am trying to make use of Manager() to share dictionary between processes and tried out the following code: from multiprocessing import Manager, Pool def f(d): d['x'] += 2 if __name__ == '__main__': manager = Manager() d =…
shadowhy
  • 13
  • 3
0
votes
1 answer

Multiprocessing manager's list does not hold values after re-initializaing

I have this situation where a multiprocess manager's list needs to be re-initialized to [] to make it blank list, but after doing so it does not modify the list anymore. It does modify it locally inside definition doesnt hold those values after the…
MohitC
  • 4,541
  • 2
  • 34
  • 55
0
votes
1 answer

Properly pickling and unpickling proxy objects in Python (multiprocessing)? "ConnectionRefusedError: [Errno 111] Connection refused while unpickling"

I am getting "ConnectionRefusedError: [Errno 111] Connection refused" while doing pickle.load() What does than actually mean? How do I properly unpickle such object? I know that the object I am trying to unpickle contains some useful data but I…
Borko Rastović
  • 76
  • 7
1 2 3 4
5
6