Common C pattern, but expressed in C++?

Question

A common pattern in C programming involves a variable length structure such as:

typedef struct {
    int length;
    char data[1];
} MyBuffer;

where data isn't literally an array of [1]. Instead, its variable length is defined by length.

The structure is allocated like:

MyBuffer* pBuff = malloc(sizeof(MyBuffer) + 100);

I want to use the same pattern, but in C++ code, so using new/delete instead of malloc/free

Can this same pattern be used in C++ code? How?

EDIT Since several answers and comments are suggesting I switch to std::vector:

I am supplied with the structure definition MyBuffer from a 3rd party, C library.
In my C++ app, I need to allocate the buffer and call a function in the C library.

On "my side" of the boundary, I prefer to keep things C++, and allocate this structure the C++-way, but I still need to pass it to a C-library that won't understand anything like a std::vector.

Answer 1

If you need to maintain compatibility with the existing C code you are using, then it works with C++, pretty much unchanged (just a need to cast the return from malloc()).

#include <stdlib.h>

typedef struct {
    int length;
    char data[1];
} MyBuffer;

void f() {

    MyBuffer* pBuff = (MyBuffer *)malloc(sizeof(MyBuffer) + 100);
}

This compiles without issue using g++.

If you are concerned about managing the memory allocated my malloc() then you could create a class to manage it and expose the MyBuffer pointer via a member method, for example:

std::shared_ptr<MyBuffer> buf((MyBuffer *)malloc(sizeof(MyBuffer) + 100), free);

Which is pretty cumbersome, I'll admit...

Answer 2

This is not idiomatic in C++, nor is it needed. The language provides std::vector<unsigned char> which wraps up the size and buffer for you, or if you don't need dynamic resizing at runtime C++11 provides std::array<unsigned char>.

EDIT: The key to note here is don't allocate the vector by itself on the heap! If you put the vector by-value on the stack or within another object and properly size it at construction you'll use exactly the same number of allocations as the C-version (one). BUT you'll be using the idiomatic language feature AND prevent yourself from making a wide variety of memory errors and/or leaks.

Answer 3

I think this will do the trick, give or take a couple syntax errors.

class MyBuffer {
 // Important, this class must not have any virtual methods.
 public:
  void* operator new(size_t data_length) {
    MyBuffer* buffer = static_cast<MyBuffer>(new char[sizeof(MyBuffer) + data_length]);
    buffer->length = data_length;
    return buffer;
  }

 private:
  int length;
  char data[1];
};

Edit:

One major downside of this technique is that it's fairly common for debug builds to override the global operator new, providing runtime checks for buffer overflows and memory leaks. I'm not sure how well this would interact with non-standard implementations of the global operator new.

Answer 4

You can use a template in C++, a feature C lacks. The template parameter determines the size of the array.

template <unsigned N>
struct MyBufferTemplate {
    int length;
    char data[N];
};

In the case the N is not known at compile time, you can define some set of reasonably sized values, and perform a best fit allocation.

However, if that seems too wasteful of memory, then another approach is to define an interface over std::vector<int> (this is based on a comment by MadScienceDreams).

struct MyBufferAdapter {
    MyBufferAdapter (int databytes = 1)
        : buf_(1 + (databytes+sizeof(int))/sizeof(int)) { buf_[0] = databytes; }
    void resize (int newdatabytes) {
        int newlength = 1 + (newdatabytes+sizeof(int))/sizeof(int);
        buf_.resize(newlength);
        buf_[0] = newdatabytes;
    }
    int & length () { return buf_[0]; }
    int length () const { return buf_[0]; }
    char * data () { return static_cast<char *>(&buf_[1]); }
    const char * data () const { return static_cast<const char *>(&buf_[1]; }
    operator MyBuffer * () { return reinterpret_cast<MyBuffer *>(&buf_[0]); }
    operator const MyBuffer * () const {
        return reinterpret_cast<const MyBuffer *>(&buf_[0]);
    }
private:
    std::vector<int> buf_;
};

Answer 5

Personally, I'd go with malloc and free. However, you could go all out with new[], placement new and delete[]:

#include <new>

struct MyBuffer {
    int length;
    char data[1];
};

MyBuffer* make_a_buffer(int size)
{
    // allocate buffer large enough for what we want
    char* raw_memory = new char[sizeof(MyBuffer) + size];

    // call placement new to put a MyBuffer in the raw memory
    MyBuffer* buffer = new (raw_memory) MyBuffer;
    buffer->length = size;
    return buffer;
}

void destroy_a_buffer(MyBuffer* buffer)
{
    // in this case, MyBuffer has a trivial (default) destructor, so this isn't
    // really needed, but in other cases you may need to call the
    // destructor
    //
    // NOTE: there is placement new, but no placement delete
    // this is the only way to correctly destroy the object
    buffer->~MyBuffer();

    // we've destroyed the object, and now we need to release the
    // memory, luckily we know we got it from new[], so we can
    // delete[] it
    delete[] static_cast<char*>(static_cast<void*>(buffer));
}

Answer 6

If you're bound to using the C struct, but want to use a better approach in C++, you can use a combination of templates and inheritance:

#include <iostream>
#include <memory>
#include <stdlib.h>

// Here's your C struct.
// Old C-style usage would be:  MyBuffer* pBuff = malloc(sizeof(MyBuffer) + 100);
// Which effectively gives you a 101-byte array for 'data'.
// (1 for the original array, +100).
typedef struct {
    int length;
    char data[1];
} MyBuffer;

// This defines a generic template that inherits from whatever you want, and
// adds some padded space to the end of it.  The 'extra_bytes' is equivalent
// to the '+100' you used to do in the c-style malloc trick (i.e. this still
// allocates a 101-byte array for 'data').
template<typename T, size_t extra_bytes>
struct ExtraBytes : public T {
  char padding[extra_bytes];
};

// If you just want to wrap your one struct, you can use this.  The constructor
// even sets the length for you.
template<size_t array_size>
struct MyBufferWrapper : public MyBuffer {
  char padding[array_size - 1];  // 1 is already allocated to 'data'
  MyBufferWrapper() {
    length = array_size;
  }
};

int main(int, char**) {
  MyBuffer normal;
  std::cout << "Sizeof normal MyBuffer = " << sizeof(normal) << "\tlength = "
         << normal.length << "\n";  // length is uninitialized

  MyBuffer* pBuff = static_cast<MyBuffer*>(malloc(sizeof(MyBuffer) + 100));
  std::cout << "Sizeof malloc'd MyBuffer = " << sizeof(*pBuff) << "\tlength = "
         << pBuff->length << "\n";  // length is uninitialized

  ExtraBytes<MyBuffer, 100> extra_bytes;
  std::cout << "Sizeof templated ExtraBytes = " << sizeof(extra_bytes)
         << "\tlength = " << extra_bytes.length << "\n";  // length is uninitialized

  MyBufferWrapper<100> wrapper;
  std::cout << "Sizeof Wrapped MyBuffer = " << sizeof(wrapper)
         << "\tlength = " << wrapper.length << "\n";  // length is set to 100

  // If you reall  auto heap = std::make_shared<MyBufferWrapper<100>>();
  auto heap = std::make_shared<MyBufferWrapper<100>>();
  std::cout << "Sizeof heap-allocated Wrapper = " << sizeof(*heap)
         << "\tlength = " << heap->length << "\n";  // length is 100

  return 0;
}

Note that with this approach, you don't need to use malloc/free nor new/delete. You just declare your MyBufferWrapper with whatever arraysize you want, it gets allocated on the stack, and you use it (you can treat it like a normal MyBuffer). If you want to use heap-allocated memory, you can just use std::unique_ptr or std::shared_ptr.

Answer 7

One C++ "ish" way to tackle this is to describe the buffer itself as a "trivially-copyable" (C++11 lingo, was "POD" for "plain old data" in C++98 & 2003) struct, with the microscopic exception that it has a private contructor to prevent instantiation. Then construct a pointer object for that struct. Here's a complete-but-trivial program with that idea:

#include <cstdlib>
#include <cstring>

struct MyBuffer
{
    int length;
    char data[1];
private:
    MyBuffer() {}
    MyBuffer& operator =(MyBuffer& other) { return other; }
};

class MyBufferPointer
{
    MyBuffer *bufptr_;

    static std::size_t getsize(std::size_t array_size)
    {
        return sizeof (MyBuffer) + array_size * sizeof (char);
    }

    static MyBuffer *getbuf(std::size_t array_length)
    {
        std::size_t sz = getsize(array_length);
        return static_cast<MyBuffer*>( malloc(sz) );
    }

public:
    MyBufferPointer() { bufptr_ = NULL; }

    MyBufferPointer(std::size_t array_length)
    {
        bufptr_ = getbuf(array_length);
        bufptr_->length = array_length;
    }

    MyBufferPointer(const MyBufferPointer &other)
    {
        const MyBuffer *op = other.bufptr_;
        if (op == NULL)
        {
            bufptr_ = NULL;
        }
        else
        {
            bufptr_ = getbuf(op->length);
            bufptr_->length = op->length;
            std::size_t sz = op->length * sizeof op->data[0];
            std::memmove( bufptr_->data, op->data, sz );
        }
    }

    MyBufferPointer& operator =(const MyBufferPointer &other)
    {
        const MyBuffer *op = other.bufptr_;
        if (op == NULL)
        {
            bufptr_ = NULL;
        }
        else
        {
            bufptr_ = getbuf(op->length);
            bufptr_->length = op->length;
            std::size_t sz = op->length * sizeof op->data[0];
            std::memmove( bufptr_->data, op->data, sz );
        }
        return *this;
    }

    ~MyBufferPointer() { if (bufptr_) free(bufptr_); }

    std::size_t size() const
    {
        return bufptr_ ? bufptr_->length : 0;
    }

    // conventience operations for access to the data array:
    char &operator [](std::size_t index) { return bufptr_->data[index]; }
    char at(size_t index) const { return bufptr_->data[index]; }
    MyBuffer* c_buffer() { return bufptr_; }
};

#include <iostream>
using namespace std;

int main()
{
    MyBufferPointer bufp;
    cout << "bufp().size() = " << bufp.size()
         << ", c_buffer=" << bufp.c_buffer() << endl;

    bufp = MyBufferPointer(100);
    cout << "bufp().size() = " << bufp.size()
         << ", c_buffer=" << bufp.c_buffer() << endl;
    return 0;
}

The MyBuffer struct is the layout for the C data area, only with private constructor and assignment operator declarations to prevent instantiation or attempt to copy (neither of which will work properly, either in C or C++.) The MyBufferPointer class encapsulates that as a C++ style char[] array, overloading the [] operator.

This still uses malloc(), not new. The memory image needed to satisfy those C APIs you mentioned needs the variable-length struct, and you can't get that in a standard C++ class created by new. This just provides a C++ wrapper to give a single point of struct creation in that class (in the static member functions getsize() and getbuf()); and guaranteed deletion of the buffer when the pointer goes out of scope. You could add resize(), to_string(), substring() or whatever methods you want.

The performance should be identical to the C struct accessed by an ordinary pointer, after optimization, since the methods are declared in-class and simple enough to be inlined.