Adding an allocator to a C++ class template for shared memory object creation

Question

In short, my question is: If you have class, MyClass<T>, how can you change the class definition to support cases where you have MyClass<T, Alloc>, similar to how, say, STL vector provides.

I need this functionality to support an allocator for shared memory. Specifically, I am trying to implement a ring buffer in shared memory. Currently it has the following ctor:

template<typename ItemType>
SharedMemoryBuffer<ItemType>::SharedMemoryBuffer( unsigned long capacity, std::string name )

where ItemType is the type of the data to be placed in each slot of the buffer.

Now, this works splendid when I create the buffer from the main program thus

SharedMemoryBuffer<int>* sb;
sb = new SharedMemoryBuffer<int>(BUFFER_CAPACITY + 1, sharedMemoryName);

However, in this case the buffer itself is not created in shared memory and so is not accessible to other processes. What I want to do is to be able to do something like

typedef allocator<int, managed_shared_memory::segment_manager>  ShmemAllocator;
typedef SharedMemoryBuffer<int, ShmemAllocator> MyBuffer;

managed_shared_memory segment(create_only, "MySharedMemory", 65536);
const ShmemAllocator alloc_inst (segment.get_segment_manager());
MyBuffer *mybuf = segment.construct<MyBuffer>("MyBuffer")(alloc_inst);

However, I don't know how to go about adding an explicit allocator to the class template.

I don't quite understand what you are asking. Do you need to know how to add a template parameter to your class template (in which case you probably need to show the current definition of the class), or do you just need to know how to construct an object at a given location in memory (in which case you should search for _placement new_). Either way, I'd be very cautious about putting non-POD objects in shared memory. Is it possible to construct from one process and destroy from another? Are you completely sure that all the class members can handle this? — CB Bailey, Mar 11 '10 at 07:44
I guess my question can simply be paraphrased as: how can you create your object in shared memory (as opposed to creating it on current process' heap and have it *use* shared memory.) — recipriversexclusion, Mar 11 '10 at 07:44
In this case you need placement new. e.g. if `shm_addr` is a `void*` pointer to shared memory you can do `MyBuffer *pBuf = new (shm_Addr) MyBuffer;` and the new `MyBuffer` will be constructed at the given location. — CB Bailey, Mar 11 '10 at 07:46
@Charles Question came out fuzzy because my understanding of shared memory is currently shaky. I have implemented a ring buffer, which uses shared memory to store values. I would like different processes to be able to read/write to/from the buffer (actually I have one writer and many readers). — recipriversexclusion, Mar 11 '10 at 07:47

score 5 · Answer 1 · answered Mar 11 '10 at 07:48

I think that you are just looking for the standard placement new.

If shm_addr is a void* pointer to shared memory you can do:

MyBuffer *pBuf = new (shm_Addr) MyBuffer;

and the new MyBuffer will be constructed at the given location. This can work with any type of object, including templated types.

You can wrap this in a separate function if you see fit.

To destroy something created with standard placement new you need to explicitly call the destructor. This is because delete would try to de-allocate the memory as regular new allocated memory which wouldn't be a valid thing to do. This is the only time in C++ that you need to explicitly call a destructor.

pBuf->~MyBuffer();

I think this is what I was looking for. Didn't know that you can specify an addr location for `new`, wow. Thanks! — recipriversexclusion, Mar 11 '10 at 07:50

uray · Accepted Answer · 2010-03-11T11:32:28.400

what make me confuse is, why you need to allocate or create an object in SharedMemory (SHM), for example if you reserve shared memory of the size 65536 Bytes, then suppose you get your shared memory at address 0x1ABC0000, if reservation success you will have free and directly accessible memory space at 0x1ABC0000 to 0x1ABCFFFF.

then when your application need to "allocate" object in SHM of size sizeof(SHMObject), and your memory manager see that address at 0x1ABC0000+0x1A is free, your memory manager should just return 0x1ABC001A value, and mark ( 0x1ABC001A to 0x1ABC001A+sizeof(SHMObject) ) was occupied, and you just need to cast: SHMObject* shmObjectPtr = (SHMObject*)(0x1ABC001A);

and ofcourse that is assuming you have your own custom memory allocator that work on specified range of memory address.

as for template, i don't really understand how does your SHM ring buffer look like, but I've done that before using SHM, my implementation is like this: `

//memory SHM allocator
template<typename T> class ShmRingAllocator
{
    protected:
        void* baseAddress;
    public:
        ShmRingAllocator(void* baseAddress,int memSize);
        void* allocate(); //this function do what I described earlier, or you can use placement new: new (baseAddress+offset)T;
}

//some kind of shared_ptr<> that handle object in SHM, this provides mechanishm to check is the pointer still valid in shm or not
template<typname T> ShmRingObjectPtr 
{
    protected:
         T* object; //mapped address of object at current process
         ShmBuffer* shm; //every object has pointer to which SHM does this pointer pointing at
    public:
         virtual T* operator->(); //operator overload to access T object
}

class ShmBuffer //base class for all kind of SHM buffer
{
    protected:
         std::string shmName;
         void* shmBasePtr;
}

template<typename T,class A=ShmRingAllocator<T>> ShmRingBuffer : public ShmBuffer
{
    protected:
         A allocator;
    public:
         ShmRingObjectPtr<T> insert() //push one element to ring buffer
         {
              return ShmRingObjectPtr<T>((T*)this->allocator.allocate(),this);
         }
}

`

Your SHM buffer prototype is very useful, thanks! I sort of understand the way you implemented it using custom SHM allocators. What I cannot understand is how you access your ShmBuf from different processes, a writer and a reader. Say, I create the ShmBuf in the writer and insert elements into it. How does an independen reader process access that object and read data from it (you also don't seem to have a pop() member). — recipriversexclusion, Mar 11 '10 at 13:52
if you want different process to be able to write and read,I solve it by creating offset_ptr<> this is again just like shared_ptr<> but contain 64-bit pointer address (on x86 platform), first it contain shm base address, and second contain offset from the base address, by using this you can use or write a pointer inside shmBUffer that pointing to data on shmBUffer, thus all pointer inside shm will point to same data in that shmBuffer, because its using offset from their own base address in each process. and the rest is just about sync, since you have virtually exact memory space between proces — uray, Mar 12 '10 at 04:34

Adding an allocator to a C++ class template for shared memory object creation

2 Answers2

Linked