Concerning running parallel code in separate processes (unlike multithreading) and/or related to package `multiprocess` from PyPI.
Questions tagged [multiprocess]
462 questions
0
votes
1 answer
Parallel processing in Python: options and alternative
I tried joblib, however, I got stuck at setting the processor affinity as explained here (the error is shown below along with my script).
Now I want to know if there are other options or alternatives that would allow me to accomplish the same goal,…
ahmadkhalifa
- 55
- 8
0
votes
2 answers
PHP separate module - Bst Practices
I'm nub in php, I'm more specialized c++.
There my question.
I need to implement such sequence:
First process performs some calculations, store it in DB (or file)
Second process answer the user requests and pass last calculations results (stored by…
tenta4
- 304
- 2
- 16
0
votes
1 answer
Why does Python mutliprocess.Process call in application start multiple instances of self?
A GUI I've created uses multiprocess.Process function call to finish work quicker by using more than one process.
w1 = multiprocessing.Process(target = _process_worker.start)
When I run the GUI as a python script gui.py any I click the button that…
tyleax
- 1,556
- 2
- 17
- 45
0
votes
2 answers
Python - Every tasks read each names from .txt file (Multiprocessing)
So I have been working abit with multiprocessing and finally got it to work with multiprocessing - Basically now I got it to run so every tasks runs now depending on how many tasks I enter.
def main():
user_input = 0
while True:
…
Andres123
- 7
- 1
- 7
0
votes
0 answers
Interacting with persistent python processes (or multiprocesses)
I would like to do something like this:
import MyProcess
proc1 = MyProcess('python').start()
proc2 = MyProcess('bash').start()
### This would mimic what happens in a python shell
print(proc1('a=10')) ==> ___
print(proc1('a*2')) ==>…
abalter
- 9,663
- 17
- 90
- 145
0
votes
2 answers
How to use multi-processing in class?
I want to use use multiprocessing to do the following:
class myClass:
def proc(self):
#processing random numbers
return a
def gen_data(self):
with Pool(cpu_count()) as q:
data = q.map(self.proc, [_ for i…
Frankie
- 744
- 1
- 9
- 14
0
votes
1 answer
Python multiprocess
I'm getting "EOFError: EOF when reading a line", when I try to take input.
def one():
xyz = input("enter : ")
print(xyz)
time.sleep(1)
if __name__=='__main__':
from multiprocessing import Process
import time
p1 =…
user1779646
- 809
- 1
- 8
- 21
0
votes
1 answer
python multiprocessing on eclipse
I write python code on eclipse using pydev.
The code is following :
from multiprocessing import Process, Queue
import time
g_workercount = 1
def calc_step():
print('calc_step started')
q = Queue()
p_worker = []
for i in range(0,…
semenbari
- 725
- 1
- 8
- 22
0
votes
1 answer
how to register a function for a process to the shared memory touched by this process?
I want to share information between multiple processes, I know if all the processes die and the shm_unlink is not called, the shared memory will still be there. I have to manually unmapped it and call shm_unlink.
But the thing is, sometimes it could…
J.R.
- 769
- 2
- 10
- 25
0
votes
1 answer
how to deal with requests failed in scrapy?
I want to scrape information about a website. And I use the proxy IP like this:
def process_request(self, request, spider):
item = random.choice(ippool)
request.meta['proxy'] = "http://"+item['ip']
request.meta['dont_redirect'] =…
yixuan
- 375
- 2
- 17
0
votes
0 answers
Python multiprocessing on same method that deals with data
I have a Django application where I am trying to manage the data for one of my models, since the number of its table rows in the database has gotten quite unruly.
I have a model staticmethod that gets all the 'Vehicle' objects in my database, goes…
TJB
- 3,706
- 9
- 51
- 102
0
votes
2 answers
Is ARM NEON thread-safe in multi-thread, multi-process or multi-core environment?
Does ARM NEON intrinsics itself have any technique to safely work in multi-thread, multi-core and multi-process environment or not? If yes, can you please cite the documents where this problem is mentioned because I cannot find one. If no, what to…
Nguyễn Thanh Vũ
- 121
- 1
- 9
0
votes
0 answers
Multiprocessing reuse Processors
I intend to use multiprocessing to speed up my image pipeline:
Therefore, I compare a process in sequence with a multiprocessing step.
Actually the sequence process is faster than the multiprocessing step.
Correct me if I'm wrong but I think it is…
Max Krappmann
- 490
- 5
- 19
0
votes
0 answers
How to let pool.starmap support infinite sequence multiprocess (for example infinite stream coming)
I made the very simple example and see whether starmap can concurrently call add_func and iter_func which yield new argument at the same time, But unfortunately, it doesn't work
from functools import partial
from itertools import repeat
from…
machen
- 283
- 2
- 10
0
votes
2 answers
Python 3: How do i start 2 processes return the answer from whichever finishes first, and stop the other process early
I have two algorithms A and B. Both algorithms take the same string as an input, perform the same transformation on the string, and return a new string.
The transformation performed can be very CPU intensive at times, and the algorithms have…
Grant Williams
- 1,469
- 1
- 12
- 24