How to push/pop movable type to/from legacy C-based container via memcpy?

Question

Several of my objects contain unique_ptr instances as members and specify a move-constructor:

struct Foo {
    Foo(Foo&& other): someA(std::move(other.someA)), someB(other.someB){}
    unique_ptr<A> someA;
    B someB;
}

This means I can move Foo objects around, but not copy them. And whenever they go out of scope, the default destructor will invoke the destructor of the unique_ptr instances:

    void someFunc(Foo&& foo);

{
    Foo a;
    Foo b;
    someFunc(Foo()); // compiles
    someFunc(std::move(a)); // compiles -> a.someA is now nullptr
    someFunc(b); // does not compile, because copy is not allowed
} // <- b is destructed and object guarded by b.someA is deleted

Now, I want to move such an object into a container and later obtain it again. This is no problem for C++ containers, because they can handle move semantics. Unfortunately my container is provided by a C-based real time OS (FreeRTOS Queue: https://www.freertos.org/xQueueSendToBack.html). Its access functions xQueueSendToBack and xQueueReceive obtain void* pointing to the item in order to use memcpy or an equivalent to move items in and out of the container.

My (simplified) solution for a wrapper of the Queue looks like this:

template <typename T, std::size_t N>
struct FreeRtosQueue {

    bool sendToBack(T&& item) {
        if (xQueueSendToBack(this->frtosQueueHandle, &item, portMAX_DELAY) != pdTRUE) {
            return false;
        }
        return true;
    } // <- item destructor called without prior move

    T receive() {
        T item;
        xQueueReceive(this->frtosQueueHandle, &item, portMAX_DELAY);
        return item;
    }

    QueueHandle_t frtosQueueHandle;
};

Unfortunately, at the end of sendToBack the destructor of the item will be called, without having it moved (from the C++ perspective). This results in all objects guarded by unique_ptrs within the item being deleted.

How can I prevent the destruction of moveable objects within the queued item?
Do you see any problems with the receive implementation?

score 0 · Answer 1 · answered Sep 27 '18 at 14:06

When passing references to C++ objects to C code, it is generally best to pass an opaque pointer to a dynamically allocated object. This would typically be created by new. In this case you can use new with T's move-constructor to get a pointer to a dynamically allocated object, unowned by any C++ RAII objects. Then you can store a copy of this pointer itself in the FreeRTOS queue.

bool sendToBack(T &&item) {
    T *item_ptr = new T(std::move(item));
    if (xQueueSendToBack(this->frtosQueueHandle, &item_ptr, portMAX_DELAY) != pdTRUE) {
        delete item_ptr;
        return false;
    }
    return true;
}

Note that the address of item_ptr is taken, rather than item_ptr itself, as xQueueSendToBack()'s pvItemToQueue argument is a pointer to the data to copy into the queue, and raw bytes of the pointer itself are what should be stored in the queue. This also means that when creating your queue, you should pass sizeof(T *) as uxItemSize when calling uxQueueCreate().

I would recommend changing sendToBack to also support copying as well as moving by switching to a forwarding reference as follows:

template<typename ItemT>
bool sendToBack(ItemT &&item) {
    T *item_ptr = new T(std::forward<ItemT>(item));
    if (xQueueSendToBack(this->frtosQueueHandle, &item_ptr, portMAX_DELAY) != pdTRUE) {
        delete item_ptr;
        return false;
    }
    return true;
}

Finally, to receive an item from the queue, this can be done as follows:

T receive() {
    T *item_ptr;
    xQueueReceive(this->frtosQueueHandle, &item_ptr, portMAX_DELAY);
    // Copy / move the item out of the dynamically allocated object and
    // into a local object.
    T item(std::move(*item_ptr));
    // The original object is moved-from but still must be deleted.
    delete item_ptr;
    return item;
}

This could be made slightly more robust and exception safe if T doesn't have a noexcept move / copy constructor by immediately transferring ownership unique_ptr as follows:

T receive() {
    T *item_ptr;
    xQueueReceive(this->frtosQueueHandle, &item_ptr, portMAX_DELAY);
    std::unique_ptr<T> item_unique_ptr(item_ptr);
    T item(std::move(*item_ptr));
    return item;
}

It is also very important that you manually go through items in your queue and delete any remaining items in FreeRtosQueue's destructor, as the stored pointers to dynamically allocated memory are otherwise unmanaged. You also ought to disable the class's copy constructor (and optionally implement a move constructor).

score -1 · Answer 2 · answered Sep 27 '18 at 13:41

The problem is that moving via memcpy is only defined for trivially movable classes, and you try to use it on a class containing a std::unique_ptr.

But it is possible to extract the underlying pointer from a unique pointer via release, and then use that pointer to reset it into a new unique pointer via reset.

So assuming B is trivially copyable/movable, you could use an ancilliary class

struct CFoo {
    A* someA;         // this is a trivial type
    B someB;          // valid provided B is trivially movable
};

You can then build a FreeRTOS queue of CFoo

template

struct FreeRtosQueue {

    bool sendToBack(Foo&& item) {
        CFoo cf { item.someA.get(), std::move(item.someB) };
        if (xQueueSendToBack(this->frtosQueueHandle, &cf, portMAX_DELAY) != pdTRUE) {
            return false;
        }
        return true;
    } // <- no destructor called here because somaA and someB have been "moved"

    Foo receive() {
        Foo item;
        CFoo cf;
        xQueueReceive(this->frtosQueueHandle, &cf, portMAX_DELAY);
        item.someA.reset(cf.someA);       // Ok, someA owns back the A object
        item.someB = std::move(cf.someB)
        return item;
    }

    QueueHandle_t frtosQueueHandle;
};

How to push/pop movable type to/from legacy C-based container via memcpy?

2 Answers2