Stack Overflow
Stack Overflow

Questions tagged [shared-memory]

Memory that may be simultaneously accessed by multiple programs

Shared memory is memory that may be simultaneously accessed by multiple programs with an intent to provide communication among them or avoid redundant copies. Depending on context, programs may run on a single processor or on multiple separate processors.

A shared memory is an extra piece of memory that is attached to some address spaces for their owners to use. As a result, all of these processes share the same memory segment and have access to it. Consequently, race conditions may occur if memory accesses are not handled properly. The following figure shows two processes and their address spaces. The yellow rectangle is a shared memory attached to both address spaces and both process 1 and process 2 can have access to this shared memory as if the shared memory is part of its own address space. In some sense, the original address spaces is "extended" by attaching this shared memory.

3133 questions
161
votes
6 answers

How to use shared memory with Linux in C

I have a bit of an issue with one of my projects. I have been trying to find a well documented example of using shared memory with fork() but to no success. Basically the scenario is that when the user starts the program, I need to store two values…
bleepzter
  • 9,607
  • 11
  • 41
  • 64
155
votes
4 answers

Shared-memory objects in multiprocessing

Suppose I have a large in memory numpy array, I have a function func that takes in this giant array as input (together with some other parameters). func with different parameters can be run in parallel. For example: def func(arr, param): # do…
CuriousMind
  • 15,168
  • 20
  • 82
  • 120
104
votes
5 answers

Shared memory in multiprocessing

I have three large lists. First contains bitarrays (module bitarray 0.8.0) and the other two contain arrays of integers. l1=[bitarray 1, bitarray 2, ... ,bitarray n] l2=[array 1, array 2, ... , array n] l3=[array 1, array 2, ... , array n] These…
FableBlaze
  • 1,785
  • 3
  • 16
  • 21
101
votes
6 answers

Share Large, Read-Only Numpy Array Between Multiprocessing Processes

I have a 60GB SciPy Array (Matrix) I must share between 5+ multiprocessing Process objects. I've seen numpy-sharedmem and read this discussion on the SciPy list. There seem to be two approaches--numpy-sharedmem and using a multiprocessing.RawArray()…
Will
  • 24,082
  • 14
  • 97
  • 108
89
votes
1 answer

Efficiently applying a function to a grouped pandas DataFrame in parallel

I often need to apply a function to the groups of a very large DataFrame (of mixed data types) and would like to take advantage of multiple cores. I can create an iterator from the groups and use the multiprocessing module, but it is not efficient…
user2303
  • 1,213
  • 10
  • 11
75
votes
2 answers

Linux shared memory: shmget() vs mmap()?

In this thread the OP is suggested to use mmap() instead of shmget() to get shared memory in Linux. I visited this page and this page to get some documentation, but the second one gives an obscure example regarding mmap(). Being almost a newbie, and…
BowPark
  • 1,340
  • 2
  • 21
  • 31
69
votes
4 answers

Combine Pool.map with shared memory Array in Python multiprocessing

I have a very large (read only) array of data that I want to be processed by multiple processes in parallel. I like the Pool.map function and would like to use it to calculate functions on that data in parallel. I saw that one can use the Value or…
Jeroen Dirks
  • 7,705
  • 12
  • 50
  • 70
58
votes
10 answers

What's the difference between the message passing and shared memory concurrency models?

Correct me if I'm wrong, but I'm surprised this hasn't been asked before on here ...
blank
  • 17,852
  • 20
  • 105
  • 159
46
votes
1 answer

When to use Pipes vs When to use Shared Memory

I am reading about various IPC mechanism. I am trying to figure out the scenarios, where we use Shared Memory and where we use named Pipes(FIFO). Pipes: Multiple process can Write, however only one process can read. Write operation is atomic. Shared…
vamsi
  • 1,727
  • 3
  • 22
  • 25
45
votes
4 answers

Use shared GPU memory with TensorFlow?

So I installed the GPU version of TensorFlow on a Windows 10 machine with a GeForce GTX 980 graphics card on it. Admittedly, I know very little about graphics cards, but according to dxdiag it does have: 4060MB of dedicated memory (VRAM) and; 8163MB…
User1291
  • 7,664
  • 8
  • 51
  • 108
44
votes
3 answers

Does using .reset() on a std::shared_ptr delete all instances

I'm new to shared_ptr's and I'm trying to figure out the exact functionality of the .reset() function. #include #include using namespace std; class SomeClass{}; int main() { shared_ptr sp (nullptr); //do some…
zeus_masta_funk
  • 1,388
  • 2
  • 11
  • 34
42
votes
2 answers

When/why use an MVar over a TVar

I find TVar's quite easy to work with even though MVar's appear a little simpler, while TVar's a little more featureful. So my question is pretty simple, what condition do I want to go to MVar rather than TVar? I suppose anytime I don't need…
Jimmy Hoffa
  • 5,909
  • 30
  • 53
40
votes
2 answers

Why do I need a memory barrier?

C# 4 in a Nutshell (highly recommended btw) uses the following code to demonstrate the concept of MemoryBarrier (assuming A and B were run on different threads): class Foo{ int _answer; bool complete; void A(){ _answer = 123; …
hackerhasid
  • 11,699
  • 10
  • 42
  • 60
39
votes
11 answers

Delete all SYSTEM V shared memory and semaphores on UNIX-like systems

How can I delete all not used semaphores and shared memory with a single command on a UNIX-like system, e.g., Ubuntu?
simone
  • 801
  • 2
  • 13
  • 25
35
votes
6 answers

wait and notify in C/C++ shared memory

How to wait and notify like in Java In C/C++ for shared memory between two or more thread?I use pthread library.
Sajad Bahmani
  • 17,325
  • 27
  • 86
  • 108
1
2 3
…
99 100