Stack Overflow
Stack Overflow

Questions tagged [multiprocess]

Concerning running parallel code in separate processes (unlike multithreading) and/or related to package `multiprocess` from PyPI.

462 questions
3
votes
1 answer

Python nose - running multiprocess programmatically

I can't run nose with several processes programmatically. This works...: PYTHONPATH="/home/developer/Downloads/unittest2-0.5.1:" nosetests --processes=4 It spawns 4 browsers at once. When running this in eclipse however, it runs them one by…
dgrandes
  • 1,187
  • 2
  • 14
  • 28
3
votes
4 answers

Executing C++ program on multiple processor machine

I developed a program in C++ for research purpose. It takes several days to complete. Now i executing it on our lab 8core server machine to get results quickly, but i see machine assigns only one processor to my program and it remains at 13%…
DataMiner
  • 325
  • 1
  • 3
  • 16
3
votes
1 answer

Prometheus how to expose metrics in multiprocess app with start_http_server

How expose metrics in multiprocess app use start_http_server I found many examples with gunicorn in internet but i want use start_http_server what should i do with code below to make it work properly? from multiprocessing import Process import time,…
Pavel Zolotov
  • 33
  • 1
  • 5
3
votes
1 answer

Use multiprocess with API requests and multiple for loops on Python

I'm accessing an API to get specific public budget from Brazil. It needs to define year, month and page. I was successful to use for loops to get the info I want for the year of 2020, looping through months {j} and pages (str +1). How can I…
Guilherme Giuliano Nicolau
  • 143
  • 1
  • 11
3
votes
2 answers

Shared pthread_cond_broadcast stuck in futex_wait

I have one "server" process a and potentially multiple "client" processes b. The server creates a shared memory file (shm_open) containing a pthread_mutex_t and a pthread_cond_t that it uses for broadcasting to the clients that something has happned…
Poohl
  • 1,932
  • 1
  • 9
  • 22
3
votes
1 answer

How to parallelize a process in a subclass in python

I am working on a project, where I have a main function, a class called simulator and a class called vehicle. In the main function I call: simulator.run() which calls vehicle.run() for all vehicles in simulator.vehicle_list[]. The vehicles have to…
Jake Lavender
  • 33
  • 3
3
votes
1 answer

Can't pickle psycopg2.extensions.connection objects when using pool.imap, but can be done in individual processes

I am trying to build an application which will "check out" a cell, which is a square covering a part of land in a geographic database, and perform an analysis of the features within that cell. Since I have many cells to process, I am using a…
wfgeo
  • 2,716
  • 4
  • 30
  • 51
3
votes
1 answer

pandas merge command failing in parallel loop - "ValueError: buffer source array is read-only"

I am writing a bootstrap algorithm using parallel loops and pandas. The problem i experience is that a merge command inside the parallel loop causes a "ValueError: buffer source array is read-only" error - but only if i use the full dataset to merge…
steveee
  • 39
  • 5
3
votes
1 answer

Worker pool data structure

I'm wondering whether there's a native implementation in the multiprocessing module that would allow me to store running processes in a list-based structure and whenever a processes is finished with execution It's automatically removed from the…
Kristijan
  • 195
  • 10
3
votes
1 answer

TypeError: can't pickle CompiledFFI objects

I am trying to get output from Telnet and SSH hosts for some commands and store them in a shelf. Since there are many commands, I am using multiprocessing. I have the following important methods: connectToHost: Making a connection (SSH/ Telnet)…
Shubham Shrivastava
  • 83
  • 1
  • 12
3
votes
1 answer

Using Gunicorn and Multiprocess with a Flask app

So I'm trying to write a Flask app that has a block of code running once every ten minutes, pulling down and processing a file. The way I currently trigger that block of code to run is a loop that sees if the delta between the current time and the…
Boyd Christiansen
  • 83
  • 2
  • 8
3
votes
2 answers

Bash script: Suppress job creation message

I have a bash function that runs a few sub-processes in parallel, like this: #!/bin/bash function check() { set +m for f in foo bar ; do ( if [ -f $f ] ; then ls -la $f >> all; fi ) & done wait } On sourcing and running this…
mahemoff
  • 44,526
  • 36
  • 160
  • 222
3
votes
0 answers

Nosetests start as multiprocess, but only one continues to run

I'm running nosetests -w tests/ --verbosity=3 --debug --debug-log=tests/logs/debug.log --with-xunit --xunit-file=$CIRCLE_TEST_REPORTS/nose/junit.xml --with-flaky --processes=4 --process-timeout=500 And see all 4 tests start initially (selenium…
Jordan Warbelow-Feldstein
  • 10,510
  • 12
  • 48
  • 79
3
votes
0 answers

How to share the large class member variable between the processes

I have a class that contains a large member variable. I want some computation to be applied to this variable concurrently. class A(object): def __init__(self): self.x = numpy.random.random((1e6,)) # some really large array def…
mistycheney
  • 33
  • 4
3
votes
2 answers

Understanding provider attribute "android:multiprocess" in Android Manifest

In creating a mobile application for Android, I am dealing with a SQLite database which requires a ContentProvider. The ContentProvider is used for adding, update, reading, or deleting data from the database. I read…
user3151285
  • 33
  • 5
1 2 3
…
30 31