11

Due to policy where I work, I am unable to use a version of Boost newer than 1.33.1 and unable to use a version of GCC newer than 4.1.2. Yes, it's garbage, but there is nothing I can do about it. Boost 1.33.1 does not contain the interprocess library.

That said, one of my projects requires placing an std::map (or more likely an std::unordered_map) in to shared memory. It is only written/modified ONE TIME when the process loads by a single process (the "server") and read by numerous other processes. I haven't done shared memory IPC before so this is fairly new territory for me. I took a look at shmget() but it would appear that I can't continually use the same shared memory key for allocation (as I assume would be needed with STL container allocators).

Are there any other NON-BOOST STL allocators that use shared memory?

EDIT: This has been done before. Dr. Dobbs had an article on how to do this exactly back in 2003, and I started to use it as a reference. However, the code listings are incomplete and links to them redirect to the main site.

EDIT EDIT: The only reason I don't just re-write Boost.Interprocess is because of the amount of code involved. I was just wondering if there was something relatively short and concise specifically for POSIX shared memory that I could re-write from scratch since data transfers between networks are also subject to a multi-day approval process...

einpoklum
  • 118,144
  • 57
  • 340
  • 684
jvstech
  • 844
  • 1
  • 9
  • 28
  • Ouch. It sounds like you are asking for trouble, I would not place an `std::map` in shared memory, boost or no boost. Are you sure this is the best way of pulling this off? A cleaner message-passing solution, perhaps? – Mahmoud Al-Qudsi Sep 27 '12 at 02:10
  • Well, I'm not sure. The software I'm converting was originally written in Python and uses the multiprocessing library to store a dictionary in shared memory. I tried looking at the C implementation of the library and couldn't figure out how the dictionary proxy was actually using shared memory, but a lot of that had to do with poor organization and me giving up the search. It should also be noted that this map is a write-once, read-only construct. It loads default values from a config file and stores them in memory until the process is restarted. There are no changes made to the values after. – jvstech Sep 27 '12 at 02:26
  • Do you have a reasonable expectation on the ceiling size of this map? (i.e. number of elements and space occupied therein ?) – WhozCraig Sep 27 '12 at 02:56
  • It's a `std::map >` which may expand as additions and plugins for the software are written. The fact that it uses a `boost::variant` value type that can contain an `std::string` may present other issues, but I can work around that if I have to. – jvstech Sep 27 '12 at 03:12
  • What I mean by "may expand" is that the number of items in the map; not the number of types available via the variant. – jvstech Sep 27 '12 at 03:22
  • 5
    So you can't use a recent Boost version but will be allowed any random open source library as long as it contains needed functionality? I smell a rat here... Anyway. Objects that contain pointers cannot be placed in shared memory. You need relative memory offsets. Boost IPC lib does not allow you to place an `std::map` in shared memory (nothing will), it offers its own version of `map`. You need to create your own. As your data structure is WORM, you can just use a sorted array. No pointers. – n. m. could be an AI Sep 27 '12 at 03:23
  • 1
    @n.m. I can't use just any open source library. We're limited by [this document](http://iase.disa.mil/stigs/os/unix/red_hat.html). And I'm aware that pointers can't be used. That was why I wanted an STL allocator so I could use it for strings, too, if needed. As far as not being able to place STL containers in shared memory, [Dr. Dobbs did it in 2003](http://www.drdobbs.com/creating-stl-containers-in-shared-memory/184401639) but their archived code listing is incomplete. – jvstech Sep 27 '12 at 03:31
  • And why do you think no one was able to replicate their success ever since? (Their method didn't work in the first place, that's why.) – n. m. could be an AI Sep 27 '12 at 03:51
  • What about it didn't work? Is there something about the internal workings of `std::map` that prevents it from happening? I would assume a placement `new` and and shared memory allocator for `std::string` _and_ `std::map/unordered_map` would force segments to be created inside shared memory without leaving behind any hanging protected pointers. Are STL containers not forced to use their supplied allocators for memory? I'm not trying to sound like a cynic or skeptic; I'm genuinely curious. – jvstech Sep 27 '12 at 04:30
  • So I have one more question. is it feasible that the map can exist in unshared memory, but their content can exist in the shared block? I.e, can you compute all the data you need to store in the pool, fashion a congruent storage of it indexible by offsets, and build the maps in each process using items that in and of themselves contain the shared pool base + offsets to their data? Assuming they all use the same comparator, they will all construct the same map with shared data assuming they're all primed correctly once started. Would *that* fill the bill of goods you need ? – WhozCraig Sep 27 '12 at 05:32
  • @WhozCraig: I don't believe that would work because the values need to be accessible by name and because of the dynamic nature of the possible consuming processes. I'm considering using a message queue or pipes at this point to send "get" messages to the server. =/ It's not something I'd want to implement. – jvstech Sep 27 '12 at 06:59
  • That would be a bummer. I think it would still be possible, but the relevant keys would have to be in the same boat, content shared in memory and indexed by id. Anyway you look at it, it is not a fun problem to solve, but makes for a helluva good question. =P – WhozCraig Sep 27 '12 at 07:01
  • Actually, yes, *that* was what I was considering at first and forgot about it until you just mentioned it: serializing the keys and data to shared memory with length prefixes. – jvstech Sep 27 '12 at 07:06

2 Answers2

7

Pointers do not work in shared memory unless you cannot pin down the shared memory at a fixed address (consistent in all processes). As such, you need specific classes that will either be contiguous (no pointer), or have an offset (and not a pointer) into the memory area in which the shared memory is mapped.

We are using shared memory at work in a pretty similar situation: one process computes a set of data, places it in shared memory, and then signal the other processes that they may map the memory into their own address space; the memory is never changed afterwards.

The way we go about it is having POD structures (*) (some including char xxx[N]; attributes for string storage). If you can actually limit your strings, you are golden. And as far as map goes: it's inefficient for read-only storage => a sorted array performs better (hurray for memory locality). So I would advise going at it so:

struct Key {
    enum { Size = 318 };
    char value[Size];
};

struct Value {
    enum { Size = 412 };
    enum K { Int, Long, String };
    K kind;
    union { int i; long l; char string[Size]; } value;
};

And then simply have an array of std::pair<Key, Value> that you sort (std::sort) and over which you use std::lower_bound for searches. You'll need to write a comparison operator for key, obviously:

bool operator<(Key const& left, Key const& right) {
    return memcmp(left.value, right.value, Key::Size) < 0;
}

And I agree that the enum + union trick is less appealing (interface wise) than a boost variant... it's up to you to make the interface better.

(*) Actually, a pure POD is not necessary. It's perfectly okay to have private attributes, constructors and copy constructors for example. All that is needed is to avoid indirection (pointers).

Matthieu M.
  • 287,565
  • 48
  • 449
  • 722
  • I don't foresee limiting string size being an issue. As to your first point, I thought that was the point of using an STL allocator in this case: allocate with `shmget()` and then use the same memory key with `shmat()` in consumer processes to map their own memory to the shared memory. You're implying this wouldn't work? – jvstech Sep 27 '12 at 07:54
  • 1
    @jvstech: I am no expert, so take those advices with a grain of salt, however Boost.Interprocess goes to great lengths to "emulate" regular pointer semantics with a combination of base-pointer (the beginning of the segment) + offset, but this requires using dedicated pointers classes (see [`offset_ptr`](http://www.boost.org/doc/libs/1_51_0/doc/html/interprocess/quick_guide.html#interprocess.quick_guide.qg_offset_ptr) for example). I am not sure than *just* specifying an allocator in a Standard container would get you there. – Matthieu M. Sep 27 '12 at 08:06
  • @jvstech: in particular, I advise that you read about the [limitations](http://www.boost.org/doc/libs/1_51_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.mapped_region_object_limitations). – Matthieu M. Sep 27 '12 at 08:07
  • My biggest worry was that some STL implementations would simply *ignore* their specified allocator and use memory however they pleased. – jvstech Sep 27 '12 at 08:08
  • Anyhow, I have basically relegated myself to something similar to this. An allocator would be great, but until then, using this will be quicker than both waiting for authorization for a newer boost install *and* getting ahold of the full code from the Dr. Dobbs article. – jvstech Sep 27 '12 at 08:15
  • @jvstech: I sympathize with you, although we have recent versions of Boost at work (well, it took a few years to get the update process approved !!), I am still stuck with gcc 4.3.2 for the foreseeable future :( – Matthieu M. Sep 27 '12 at 08:28
0

Simple workaround. Create your own "libNotBoost v1.0` from Boost 1.51. The Boost library allows this. Since it's no longer Boost, you're fine.

MSalters
  • 173,980
  • 10
  • 155
  • 350
  • I took at look at the interprocess lib from 1.39.0 (which I'm *trying* to get an exception over policy so I can get it installed) and decided that might actually be just as much work as implementing the entire base of the software I'm converting... – jvstech Sep 27 '12 at 07:10
  • I've tried to extract pieces of boost before. Though boost does provide you with a tool that helps, it's not only a lot of work, but you inevitably end up with dozens if not hundreds of dependent files. – Tom Swirly Sep 27 '12 at 17:21