Questions tagged [multiprocessing-manager]
88 questions
0
votes
1 answer
Sharing a custom class object through multiprocessing manager
I am working on a project where I have very specific kind of logs and for this logs to work I have created a class which basically maintains a dictionary that logs time and actions taken for each step of the execution. Which is later used for…
Eternal
- 928
- 9
- 22
0
votes
1 answer
How to convert ProxyObject in python multiprocessing manager to normal python object
I'm using the multiprocessing library in python 3.7.
In my use case, I start with a normal list of dicts - I use this to create a ListProxy object from a SyncManager which I then use in a pool to syncronise the list (created as below).
data =…
elembie
- 600
- 2
- 8
0
votes
1 answer
pandas + multiprocessing: "NotImplementedError: Not supported for DataFrames!"
After my previous thread was marked as a duplicate, it pointed me in the direction of multiprocessing managers. I'm trying to use multiprocessing to create a service that handles my pandas dataframe to give to Flask requests. This is my code so…
jarthur
- 393
- 1
- 2
- 18
0
votes
1 answer
Suggestions on using python multiprocessing to get numpy array returned from target function?
I am completely new to python multiprocessing and a bit overwhelmbed by the vast amount of online resources, so I kinda wanna a more clear-cut approach from here. My code looks like the following: the two functions, forward and backward are very…
Yilei Huang
- 77
- 1
- 7
0
votes
0 answers
multiprocessing.Manager().Queue() TypeError: can't pickle thread.lock objects
I assign an instance object (q) of the multiprocessing.Manaager().Queue() to one item of the instance object (d) of the multiprocessing.Manager().dict(), but I cannot reuse the q in the dict.
The version of the Python is 2.7.15.
#!/usr/bin/env…
Congming SHI
- 1
- 1
0
votes
0 answers
multiprocessing.process in python for parallel processes consuming values from shared queue
I am trying to utilize python multiprocessing pool to take values from a shared queue and consume in worker processes.
import multiprocessing
def myTask(queue):
while not queue.empty():
value = queue.get()
print "Process {}…
Dania
- 1,007
- 1
- 14
- 21
0
votes
1 answer
Python multiprocessing Manger OSError "Only one usage of each socket address"
I am creating a communication platform in python (3.4.4) and using the multiprocessing.managers.BaseManager class. I have isolated the problem to the code below.
The intention is to have a ROVManager(role='server') instance running in one process on…
Martin Løland
- 1
- 3
0
votes
1 answer
python manager register issue or just my faulty code
I use a python manager to share data between processes. I wanted to break up my code into smaller pieces and created a function 'soFactory' that would take a dictionary of name/object pairs and register the object with the manager under the given…
Adrian Cepleanu
- 57
- 1
- 4
0
votes
2 answers
Modify a list in Multiprocessing pool's manager dict
I have a list of elements which I am processing in a multiprocessing apply_async task and updating elements processed one by one with a key in manager dict on which I want to map whole list.
I tried following code:
#!/usr/bin/python
from…
MohitC
- 4,541
- 2
- 34
- 55
-1
votes
1 answer
How to run parallel processes with multiprocessing terminate one of them from another and then restart
I get error when running the code
TypeError: Pickling an AuthenticationString object is disallowed for
security reasons
I am very new to this... You can add your own completely new solution/code to this similar to my code as long as you can meet…
Ggjustice V
- 75
- 8
-1
votes
1 answer
Pool apply function hangs and never executes
I am trying to fetch Rally data by using its python library pyral. Sequentially the same code works, but its slow.
I thought of using python multiprocess package, however my pool.apply method gets stuck and never executes. I tried running it in…
Hamzah Jamal
- 59
- 6
-1
votes
1 answer
Calling threading.get_ident() returns same ID between different threads
While I was learning to use SyncManager sharing common object among multiple processes, I noticed a odd behavior of Thread working in the context of SyncManager.
To my understanding, SyncManager creates a server process, accepting connections from…
Kamoo
- 832
- 4
- 11
-2
votes
1 answer
Value is not storing into Manager().dict() inside the Pool()
This is the main code:
if __name__ == '__main__':
manager = Manager()
mylist = manager.list()
mydict = manager.dict()
mylist.append('abc')
mylist.append('xyz')
for k in mylist:
mydict[k] = manager.list()
pool =…
aste123
- 1,223
- 4
- 20
- 40