2

This is regarding the applicability or suitability of message queue vs. Shared Memory in this situation:

  1. multiple DLLs or shared libraries

  2. each library will try to communicate with My main application DLL or shared library, e.g., the I/P to and O/P from all the DLLs or Shared libraries are to be communicated through my main application's Shared library. These communications are ASYNCHRONOUS.

  3. some of the DLLs or shared libraries, other than my application's .so, will create multiple threads, and output from each such thread needs to be communicated back to my application library. The output from these threads are again ASYNCHRONOUS.

  4. my main application DLL / .so will continue with its other work which is very likely that it communicates with some server over the network and it responds accordingly

  5. The functioning of all other DLLs/ .so's are asynchronous

Q-1: In the above situation which is the best fit ? (I) a message queue , (II) a shared memory ?

Q-2: Any reference or link which enforces synchronization between several shared libraries using shared memory ?

MSalters
  • 173,980
  • 10
  • 155
  • 350
ind79ra
  • 149
  • 2
  • 13
  • regarding Q2 -(Sorry wrong link first time) http://stackoverflow.com/questions/13512170/communication-using-shared-memory-between-vc-and-qt-applications/13512505#13512505 – Caribou Nov 23 '12 at 12:01
  • 1
    regarding Q1 - Depends on the use case - you need transactional capabilities, Garenteed delivery? Use MQ, But Shared Mem is faster and requires more plumbing – Caribou Nov 23 '12 at 12:04
  • Thanks Caribou for your reply, I can not use any 3rd party software, like MQ, here. Though shared memory is faster and can provide more space as compared to message queue but the synchronization is bugging me. – ind79ra Nov 26 '12 at 07:11

2 Answers2

1

I guess your understanding of the problem is wrong:

  • Shared memory is a "channel" to connect different processes, like sockets, pipes, or normal memory.
  • A message queue is a "protocol" for passing messages, like TCP or a ring buffer. You may create it over a socket (like 0MQ) or using a synchronized queue (like Intel TBB, see below) in shared or "normal" memory.

You do not need shared memory with the specifications you give. Shared memory is useful if one of the following is true:

  • You have several processes (you don't, all your so/dlls will share the same memory)
  • You need to persist the memory of your process if it crashes (you may need that but didn't mention).

Now, you need to choose a protocol for your code to talk over it. I'd recommend using Intel Thread Building Blocks (TBB, that would answer Q2). They provide different layers of abstractions for what you want to achieve, I do not know enough to choose for you, though, take some time to read the (long) docs.

J.N.
  • 8,203
  • 3
  • 29
  • 39
  • 1. Shared memory is not a pipe. It is, er, shared memory. Like, the same memory will show up in different process' address spaces. 2. Shared memory has nothing to do with persistence. Maybe you are thinking of memory-mapped files? – Bryan Nov 23 '12 at 16:24
  • @Bryan You read my answer wayyyy too literally. The quotation marks around pipe should have hinted I didn't mean a "unix pipe". I meant "channel of communication". My bad for the wrong wording. Then, Shared memory has EVERYTHING to do with persistence. The point is that persistence is not something limited to disk. If you give a SHM region a name, it will persist across processes (imagine what would happen if it died when the first process using it crashed). – J.N. Nov 23 '12 at 16:52
0

With the caveat that I only understand your application in the most general sense, I think message queues are a no-brainer provided the msgs you are passing are bounded and appropriate for a queue to being with.

The two primary things you seem to be concerned with are synchronization and asynchronousity. (1) POSIX message queues have the queue synchronization already built-in. That's one big headache removed. (2) Under Linux, the queue id mqd_t is a file descriptor which means it can be used in a select statement. That takes care of the asynchronous concerns. In your main DLL you can load up the mqd_t descriptors for all your queues in select statement and process your DLL msgs as they arrive with a consistent, debugged and well understood mechanism. (3) The tiny bit (and it is tiny for most apps) of efficiency you lose compared to shared memory is more than made up by the relative ease of msg queue use and the fact that your main application DLL is going to spend a relatively long time waiting on network I/O with the server anyway.

Duck
  • 26,924
  • 5
  • 64
  • 92
  • Thanks Duck, you are right in saying that i am worried about the synchronization and the asynchronousity. well the message queue has built in synchronization but is it a good idea to use message queue in case when multiple DLLs / shared libraries are involved ? and the amount of data can be huge at any point of time , like sometimes it can be a small string and sometimes it can be in 100 MBs or even 1 GB, so the storage capacity provided by message queue would be enough in this case ? anyway thanks again for insightful views. – ind79ra Nov 26 '12 at 07:22
  • @Indranil - no, given that kind of data a MQ is probably not appropriate. There are some measures you can take to avoid copying so much data to the queue - e.g. alloc data on the heap and just pass a ptr and length in the queue - but from you say you are probably better off with some other mechanism. – Duck Nov 26 '12 at 16:40