Stack Overflow
Stack Overflow

Questions tagged [concurrent-processing]

[CONCURRENT] processing, in sharp contrast to a [PARALLEL] processing, has no specific requirements on when to start, where & in what particular order to execute and when or how to finish a set of tasks, that got a chance to be executed in a free flowing, uncoordinated fashion on a subset of available resources.

In a "just"-[CONCURRENT] process orchestration, the code-units (be it tasks, threads, programmes... ) may but, need not, appear to get executed in some non-[SERIAL], overlapped in time, fashion or simultaneously (just by coincidence), all that by using more than one CPU or processor core and other shared resources to execute a program or multiple computational units.

A "just"-[CONCURRENT] process orchestration does not warrant any mutual coordination, the less a synchronicity a True-[PARALLEL] warrants by definition.

A "just"-[CONCURRENT] process orchestration enables more things to happen con-currently, the process-scheduler and other constraints participate in evaluating, who gets "elected" for such a con-current processing and who not.

68 questions
1
vote
1 answer

How to implement download queue async with parallel foreach

I wanna try something like download from a collection of image in a minimum of 3 concurrent download but, as soon as one of the 3 finished downloading add a new download from the list or must wait to add a new download until a download is finished…
ropenrom24
  • 517
  • 8
  • 18
1
vote
1 answer

Does slicing dataframes in multiprocess / multiprocessing improve performance?

I need to do some computation on different slices of some big dataframes. Suppose I have 3 big dataframes df1, df2 and df3. Each of which has a "Date" column. I need to do some computation on these dataframes, based on date slices and since each…
mpy
  • 622
  • 9
  • 23
1
vote
2 answers

How to wait for a signal for a certain time only, in C language?

I am trying to communicate between 2 processes ( parent and child process in linux ) using signals in C programming language. The first process does some calculation and provides a data. Then, it sends a signal to the second, which is in a suspended…
Gismail
  • 89
  • 10
1
vote
0 answers

How can I run the same python script in parallel started from a cron job?

I have a watchfolder where files are dropped. Via a cronjob I start a python script that first checks for new files. def file_to_process(): filePath = "".join([base_url_gpfs, '/*']) if glob.glob(filePath + "*.xml"): set_pid_file() …
user2811144
  • 91
  • 7
1
vote
1 answer

Running a command over too many files

I want to encrypt and decrypt big files (think 20m lines) of text. The encryption service I am using can only encrypt a maximum size of 64kb. For the purposes of this question assume we are stuck with this service. My solution is to split the huge…
pilu
  • 720
  • 5
  • 16
1
vote
1 answer

Form submissions lose idempotency when submitted rapidly in a row

I had a problem with people double clicking my form submit button and it submitting multiple times, so I wanted to allow only one submission. My first thought was javascript, but that's not absolutely instantaneous and I wanted something 100%…
Joe Morano
  • 1,715
  • 10
  • 50
  • 114
1
vote
1 answer

Non-blocking socket server running in python3 in another thread

I would like to know the best way to run a non-blocking python3 socket server. I currently have code that is: def start(data): global sock try: sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) …
Kaveen K
  • 33
  • 4
0
votes
4 answers

Getting a Chunk of Work

Recently I had to deal with a problem that I imagined would be pretty common: given a database table with a large (million+) number of rows to be processed, and various processors running in various machines / threads, how to safely allow each…
Otávio Décio
  • 73,752
  • 17
  • 161
  • 228
0
votes
1 answer

Jenkins - How to Create Dynamic Parallel Stages?

I basically have 4 curl commands to trigger a webhook and they're roughly the same except for a single argument. Therefore, I would like to extract this curl command and create a parallel stage that can iterate through each unique argument and then…
Riley Hun
  • 2,541
  • 5
  • 31
  • 77
0
votes
2 answers

Java - Execute multiple Runnables in parallel and trigger to timeout some of long running tasks?

I've a list of Runnable tasks (e.g. it contains 100 tasks, and each task takes randomly 1 - 10 seconds). The tasks must run in parallel and from the thread pool provided by ExecutorService (e.g. my system has 4 CPUs, then it should run 4 tasks at…
Bằng Rikimaru
  • 1,512
  • 2
  • 24
  • 50
0
votes
0 answers

Communication between process in c with message queue

this would be a basic version of what the project will be. I have X user that generate transaction handled by a father. Then, I've tried to establish a connection with a message queue between father and child but when i launch process child with…
Matteo Pagliarello
  • 171
  • 1
  • 10
0
votes
1 answer

How to make python program faster? Is multithreading useful here?

I have a JSON file that I am parsing through in an attempt to see if a domain is live. The code I have is the following: for i in range(len(json_data)): print(i) if int(json_data[i]['response']['result_count'])>0: for j in…
user18853421
0
votes
3 answers

Evaluating methods in parallel

Let's say I need the output from the below independent methods var x = LongCalculation(123) var y = LongCalculation2(345) var z = LongCalculation3(678) var b = LongCalculation4(910) What is the least code invasive way of executing these…
alenyb
  • 31
  • 3
0
votes
0 answers

How to run both Python scripts since the startup, launched from bash script on Raspberry Pi?

A pair of my Python scripts are, pyserial.py predict.py My bash.sh looks like : #!/bin/sh python3 path/pyserial.py & python3 path/predict.py & To run the bash on startup I edited crontab as, @reboot sh path/bash.sh & But on startup, only…
FarhanAvro
  • 11
  • 2
0
votes
0 answers

Why WSL Linux concurrent processes not fully utilizing CPU are stuck in suspended mode?

I am very new to working with Linux so please be as elaborate as possible in your answers, thanks! I am running WSL on Windows 10 using Ubuntu 18.04 LTS, and I am trying to run several (56) parallel processes in the background. From what I know,…
patrick7
  • 366
  • 1
  • 11
1 2
3
4 5