c++ making struct with unknown size array (with memory location intact) for passing buffer data

Question

So, I need the memory to be lay out in a specific way:

struct aPacketWithMoreData{
    double x;
    int16 h;
    int16 noNeedToBePassedToTheBuffer;
    double y;
}
struct aPacket{
    double x;
    double y;
    int16 h;
}
struct bufferData{
    int32 something;
    uint64 anotherthing;
    aPacket arrayOfData[]; //with unknown size or runtime size
    float32 somethingThatEndThis;
    int16 withMaybeWeirdSizeAlinement;
}

And then I hace an array of aPacketWithMoreData. I need to make a temp bufferData so that I can call the API (which is very slow) once to copy the buffer. What is the best way to do this? (Note that storing a std::vector doesn't help because a std::vector is actually just a pointer to an array, and in the other side of the API, it will not be able to de-reference that.)

I know that in c++ you can do something like this: (link: https://en.cppreference.com/w/c/language/struct)

struct s{
    int a;
    float b;
    double c[]; //array of unknown size
}
...
void function(uint numberOfC) {
    struct s objectName = malloc(sizeof (struct s) + sizeof(double)*numberOfC);
    ...
}

But I believe that this is for portabilities with c. This would also not work when the array is in the middle (required for the buffer structure). So other than hacking it like below, is there a better way?

void function(uint number) {
    char* buffer = new char[sizeof(int32)+sizeof(uint64)+sizeof(float32)+sizeof(int16)+sizeof(aPacket)*number];
    *(int32*)(buffer[0]) = returnsInt32();
    *(uint64*)(buffer[sizeof(int32)]) = returnsuint64();
    ...
    size_t offset = sizeof(int32)+sizeof(uint64);
    for (uint i=0; i<number; i++) {
        aPacketWithMoreData& obj = aVector[i]; //inore the possible out of bound for now
        *((aPacket*)(buffer[offset])+i) = aPacket(obj.x,obj.y,obj.h);
    }
    ...
    AnAPI.passData(buffer, sizeof(buffer));
}

Edit: Fixed the char array thingy.

Answer 1

That's not for compatibility with C, that's an extension outside of standard. There is no way to represent undetermined size of array, not at least defined in standard. In C or C++ type is know on compile time, which means it memory layout is known: size of type and offsets\memory locations of every element.

Some platforms like Windows have specifically tuned C++ compiler to support existing API, but that isn't guaranteed by standard, but in later versions you can't use [], instead [1] is used. Typical Windows API structure with variable length:

typedef struct _SYMBOL_INFO {
  ULONG   SizeOfStruct;   // size of structure, a tradition safety measure in API
  ULONG   TypeIndex;
  ULONG64 Reserved[2];
  ULONG   Index;
  ULONG   Size;
  ULONG64 ModBase;
  ULONG   Flags;
  ULONG64 Value;
  ULONG64 Address;
  ULONG   Register;
  ULONG   Scope;
  ULONG   Tag;
  ULONG   NameLen;
  ULONG   MaxNameLen;   // length of Name in bytes.
  CHAR    Name[1];
} SYMBOL_INFO;

Use of such structure in C++ would require something like this:

    // buffer for SYMBOL_INFO with string of 1024 bytes.
    ULONG64 buffer[ (sizeof( SYMBOL_INFO ) + TRACE_MAX_FUNCTION_NAME_LENGTH 
               + sizeof( ULONG64 ) - 1) / sizeof( ULONG64 ) ];
    
    SYMBOL_INFO *info = new (buffer) SYMBOL_INFO {};
    info->SizeOfStruct = sizeof( SYMBOL_INFO );
    info->MaxNameLen = TRACE_MAX_FUNCTION_NAME_LENGTH;

Your structure needs to be dissected in two.

struct bufferData{
    int32 something;
    uint64 anotherthing;
    uint   lengthOfArray;   // length of following array
    aPacket arrayOfData[1]; // varadic size
};
struct bufferFooter{
    float32 somethingThatEndThis;
    int16 withMaybeWeirdSizeAlinement;
};

Note that somethingThatEndThis may have alignment requirement, so bufferFooter would too.

// add number of packets to size of this buffer, by default it's one.
uint64_t *buffer  = new uint64_t[ (sizeof(bufferData)+sizeof(bufferFooter) 
    + (sizeof(uint64_t)-1))/sizeof(uint64_t) ]; 

bufferData* dptr = new (&buffer) bufferData {};
bufferData* fptr = new (&buffer[sizeof(bufferData)/sizeof(uint64_t)]) bufferFooter {}; 
dptr->lengthOfArray = 1;

this aligns structure by 64bit word.. if required alignment by char, this might get a little complicated unless memcpy is used or platform supports free alignment. For "receiving" such structure from outside source reinterpret_cast is a legitimate tool as long as structures are POD.

If those structures are part of some binary file structure or network protocol with extreme need of fast extraction of data, common practice is to add header which would include offsets of those "wandering" structures beyond variable array. Good examples would be structures of .bmp or .png file. Where extraction time isn't issue, many protocols shifted to xml-like structures, binary representation or text.