Create a shared-memory vector of strings

Question

I am trying to create a class managing a shared-memory vector of (std)strings.

typedef boost::interprocess::allocator<std::string, boost::interprocess::managed_shared_memory::segment_manager> shmem_allocator;
typedef boost::interprocess::vector<std::string, shmem_allocator> shmem_vector;

shmem_mgr::shmem_mgr() :
    shmem_(create_only, SHMEM_KEY, SHMEM_SIZE),
    allocator_(shmem_.get_segment_manager())
{
    mutex_  = shmem_.find_or_construct<interprocess_mutex>(SHMEM_MUTEX)();
    condition_ = shmem_.find_or_construct<interprocess_condition>(SHMEM_CONDITION)();
    //buffer_ is of type shmem_vector
    buffer_  = shmem_.construct<shmem_vector>(SHMEM_BUFFER_KEY)(allocator_);
}

void shmem_mgr::run() {
    running_ = true;

    while(running_) {
        scoped_lock<interprocess_mutex> lock ( *mutex_ );

        int size = buffer_->size();

        log_.debug() << size << " queued request(s) found" << std::endl; //LINE 27
        for(int i=0; i<size; i++) {
            log_.debug() << buffer_->at(i); // at() crashes my app
        }

        buffer_->clear(); //so does clear()
        condition_->wait (lock);
    }
}

The client successfully adds a string to the vector (it also succeeds to read that string from the buffer for debug), the manager (code above) receives the signal (condtion variable), writes that there is a string in the vector (line 27), but when it tries to get that string via at() the application crashes.

Edit: I've figured out, that the use of std::string is not possible, there is a string container in boost ipc just for this case. This doesn't change the fact that I need a vector of (boost/std) strings...

Q: How can I pass strings across shared memory? I need to store them in some buffer (capable of storing >1 at a time) in shmem, and then fetch in second process - that's the requirement. The input is always std::string and so is the output, but the internal representation in shmem may be different.

@elmes: Still wrong, you're passing an iterator to `at()`, but `at()` only takes a `std::vector::size_type`. Even if that code would compile (it won't) it would make no sense. Also kind of strange that `log.debug()` returns an lvalue which is meant to be used that way... why would you pass an argument? — Ed S., Oct 19 '12 at 19:08
1. Does it have to be a vector? 2. Does the shared memory need to hold the strings or is it sufficient to hold pointers to strings? 3. Do you need concurrent access (I assume not from your use of mutex)? 4. Does the code need to be cross platform? You probably know by now that the standard iterators are not thread safe. If you can answer the above questions, I can provide a solution. — Ian Thompson, Oct 22 '12 at 21:56
1. Not necessarily, but random-access yes. 2. I have a use case where on input I have std::string and I need a std::string on output (2nd process) 3. I can't forbid concurrent usages. Mutex is for conditon, and condition is to not waste cpu time (but wait for another input in the consumer process) 4. Yes! — emesx, Oct 23 '12 at 04:50
To be precise, I need to send polymorphic classes (kept via base-class pointers) over 2 processes. Boost isn't capable of this, so I serialize those objects to strings. — emesx, Oct 23 '12 at 05:03
I'm still confused what the mutex is for. You don't show the code/function that pushes the strings into the shared memory. Do you get a lock on the mutex in the 'push string' function? As for passing subclasses, can you keep the type of the sub-object in the object itself and use dynamic_cast to down cast the object in the reciever? One last question, who disposes of the 'objects'/strings? Is it the receiver? Or does the vector keep growing until it runs out of memory? — Ian Thompson, Oct 23 '12 at 09:01
yes, mutex is used in clients while adding strings, and to trigger the condition so that the consumer may react. When the receiver wakes up from `wait`ing on the condition, it will get all new strings, process them and remove from vector. Then it will `wait` again.. — emesx, Oct 23 '12 at 12:41

score 16 · Accepted Answer · answered Oct 24 '12 at 01:57

From the docs.

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/interprocess/allocators/allocator.hpp>

int main ()
{
   using namespace boost::interprocess;
   //Typedefs
   typedef allocator<char, managed_shared_memory::segment_manager>
      CharAllocator;
   typedef basic_string<char, std::char_traits<char>, CharAllocator>
      MyShmString;
   typedef allocator<MyShmString, managed_shared_memory::segment_manager>
      StringAllocator;
   typedef vector<MyShmString, StringAllocator>
      MyShmStringVector;

   //Open shared memory
   //Remove shared memory on construction and destruction
   struct shm_remove
   {
      shm_remove() { shared_memory_object::remove("MySharedMemory"); }
      ~shm_remove(){ shared_memory_object::remove("MySharedMemory"); }
   } remover;

   managed_shared_memory shm(create_only, "MySharedMemory", 10000);

   //Create allocators
   CharAllocator     charallocator  (shm.get_segment_manager());
   StringAllocator   stringallocator(shm.get_segment_manager());

   //This string is in only in this process (the pointer pointing to the
   //buffer that will hold the text is not in shared memory).
   //But the buffer that will hold "this is my text" is allocated from
   //shared memory
   MyShmString mystring(charallocator);
   mystring = "this is my text";

   //This vector is only in this process (the pointer pointing to the
   //buffer that will hold the MyShmString-s is not in shared memory).
   //But the buffer that will hold 10 MyShmString-s is allocated from
   //shared memory using StringAllocator. Since strings use a shared
   //memory allocator (CharAllocator) the 10 buffers that hold
   //"this is my text" text are also in shared memory.
   MyShmStringVector myvector(stringallocator);
   myvector.insert(myvector.begin(), 10, mystring);

   //This vector is fully constructed in shared memory. All pointers
   //buffers are constructed in the same shared memory segment
   //This vector can be safely accessed from other processes.
   MyShmStringVector *myshmvector =
      shm.construct<MyShmStringVector>("myshmvector")(stringallocator);
   myshmvector->insert(myshmvector->begin(), 10, mystring);

   //Destroy vector. This will free all strings that the vector contains
   shm.destroy_ptr(myshmvector);
   return 0;
}

Almost perfect. Just describe how to convert `MyShmString` to a `std::string`. — emesx, Oct 24 '12 at 20:42
`std::string s = std::string(myShmString.begin(), myShmString.end());` — Paul Groke, Oct 24 '12 at 21:35

score 2 · Answer 2 · answered Oct 20 '12 at 22:52

you need a custom allocator for your sharable stl classes. you need a self-based pointer (ACE & boost have these) defined in the allocator. On opposite sides the (CONTIGUOUS) shared memory typically resides at different addresses. You need a shared memory allocation subsystem(heap manager) too (that the allocator allocates from) - all non-trivial low level code, but most definitely doable and once you have it, it is usable everywhere. If you do all that, you need only pass the displacement (from beginning of the (CONTIGUOUS!!) heap area) of the non-flat structure around.

You can create queues and everything else you might want - all provided that the "pointers" in the objects are self-based and that the dis-contiguous pieces in your non- flat pieces come from one large contiguous piece.

You can't use std::string because, unless you control the allocation, the memory in standard string has NOTHING to do with your shared memory - same for any other stl structure

It is also imperative (as usual) to resolve/agree ownership issues

Ok. How about a `boost::interproces::vector` of `boost::interprocess::string` or any other vector of some strings? I need to transfer some strings across 2 processes, what do I do — emesx, Oct 21 '12 at 06:24

John Qualis · Answer 3 · 2012-10-25T09:56:59.957

You can use boost::interprocess::managed_shared_memory. The following program passes a boost::interprocess::string between 2 processes. Works fine on my machine (Ubuntu Linux). You can use managed_shared_memory to pass vectors or objects. boost::interprocess::string has a c_str() method.

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <cstring>
#include <cstdlib>
#include <string>
#include <iostream>

int main(int argc, char *argv[])
{
  using namespace boost::interprocess;
  typedef boost::interprocess::allocator<char, boost::interprocess::managed_shared_memory::segment_manager> CharAllocator;
  typedef boost::interprocess::basic_string<char, std::char_traits<char>, CharAllocator> string;
  if(argc == 1){  //Parent process

      boost::interprocess::shared_memory_object::remove("MySharedMemory");

      //Create a shared memory object.
      managed_shared_memory shm (create_only, "MySharedMemory", 1024);

      string *s = shm.find_or_construct<string>("String")("Hello!", shm.get_segment_manager());
      std::cout << *s << std::endl;

      //Launch child process
      std::string s1(argv[0]); s1 += " child ";
      if(0 != std::system(s1.c_str()))
         return 1;
  }
  else{
      //Open already created shared memory object.
      managed_shared_memory shm (open_only, "MySharedMemory");
      std::pair<string *,std::size_t> ret = shm.find<string>("String");
      std::cout << *(ret.first) << std::endl;
  }
  return 0;
}

Create a shared-memory vector of strings

3 Answers3

Linked