2

I have the following structure:

typedef struct Octree {
    uint64_t *data;
    uint8_t alignas(8) alloc;
    uint8_t dataalloc;
    uint16_t size, datasize, node0;
    // Node8 is a union type with of size 16 omitted for brevity
    Node8 alignas(16) node[]; 
} Octree;

In order for the code that operates on this structure to work as intended, it is necessary that node0 immediately precedes the first node such that ((uint16_t *)Octree.node)[-1] will access Octree.node0. Each Node8 is essentially a union holding 8 uint16_t. With GCC I could force pack the structure with #pragma pack(push) and #pragma pack(pop). However this is non-portable. Another option is to:

  • Assume sizeof(uint64_t *) <= sizeof(uint64_t)
  • Store the structure as just 2 uint64_t followed immediately by the node data, and the members are accessed manually via bitwise arithmetic and pointer casts

This option is quite impractical. How else could I define this 'packed' data structure in a portable way? Are there any other ways?

ndim
  • 35,870
  • 12
  • 47
  • 57
user16217248
  • 3,119
  • 19
  • 19
  • 37
  • Why does packing the structure with `#pragma pack(push)` in GCC matter? Unless I miscounted, with `uint64_t *` being eight bytes, all the members would be naturally aligned with no padding, so packing would not change anything. If `uint64_t *` is four bytes, there is going to be four bytes of padding before the next member, since you have `alignas(8)` but none between `node0` and `node`. Are you concerned some weird platform will insert padding? Also, how do you expect packing to interact with the explicit `alignas` requests? – Eric Postpischil Jun 11 '23 at 00:42
  • 2
    Are you aware that even if the `node` member is in the desired location, the C standard does not guarantee that accessing it with `((uint16_t *)Octree.node)[-1]` will work? Some compilers might give such a guarantee, or there are workarounds that might suit you, depending on what the actual requirements are. – Eric Postpischil Jun 11 '23 at 00:49
  • @EricPostpischil The reason I pack it, although it doesn't change anything, is to *'guarintee'* the packing. However the C standard makes no such guarintee, even if all the members are properly aligned. – user16217248 Jun 11 '23 at 00:55
  • @EricPostpischil The first `alignas()` is to ensure that the total structure size will remain 16 bytes even if `sizeof(uint64_t *) < 8`. Also what rule disallows `((uint16_t *)Octree.node)[-1]`? Does that depend on the definition of `Node8`? Should I provide that anyway? – user16217248 Jun 11 '23 at 01:24
  • 2
    Regardless of the way which tells the C compiler how to lay out the data structure in memory, you can always use a few ISO C11 `static_assert` statements with `offsetof` and `sizeof` to ensure the memory layout actually is what you expect. Otherwise, compilation will fail. You can always add more compiler specific memory layout instructions then. This might not quite be "portable C", but it is still C11 plus some compiler specific instructions for every compiler which compiles the code. If you need a properly portable memory layout for a type, you need another programming language. – ndim Jun 11 '23 at 02:00
  • @user16217248, "it is necessary that node0 immediately precedes the first node such that ((uint16_t *)Octree.node)[-1] will access Octree.node0' seems to be **the** goal. 1) is packing the members before `.node0` also a goal? It is not needed to achieve the 1st goal. Is making `.node` align to 16-byte address also a goal? That is not needed to achieve the 1st goal. – chux - Reinstate Monica Jun 11 '23 at 05:35
  • Re "*the C standard makes no such guarintee, even if all the members are properly aligned.*", Probably true, but no compiler would add padding between already-aligned members. – ikegami Jun 11 '23 at 05:54
  • 2
    Re “Also what rule disallows `((uint16_t *)Octree.node)[-1]`?”: The rule that specifies pointer arithmetic, C 2018 6.5.6 8, defines it only for arithmetic within an array (including the end position one beyond the last element and treating a single object as an array of one element). This creates a “pointer provenance” property; if `p[x]` has behavior defined by the C standard, it can refer only to elements of an array `p` points to. Compilers may use this to reduce pointer arithmetic when optimizing, and this reduction may break code that attempts to use indexing outside the actual array. – Eric Postpischil Jun 11 '23 at 09:56
  • 1
    "packed structure" and "portable C" are fundamentally incompatible concepts. – Andrew Henle Jun 11 '23 at 12:08

1 Answers1

3

The C language standard does not allow you to specify a struct's memory layout down to the last bit. Other languages do (Ada and Erlang come to mind), but C does not.

So if you want actual portable standard C, you specify a C struct for your data, and convert from and to specific memory layout using pointers, probably composing from and decomposing into a lot of uint8_t values to avoid endianness issues. Writing such code is error prone, requires duplicating memory, and depending on your use case, it can be relatively expensive in both memory and processing.

If you want direct access to a memory layout via a struct in C, you need to rely on compiler features which are not in the C language specification, and therefore are not "portable C".

So the next best thing is to make your C code as portable as possible while at the same time preventing compilation of that code for incompatible platforms. You define the struct and provide platform/compiler specific code for each and every supported combination of platform and compiler, and the code using the struct can be the same on every platform/compiler.

Now you need to make sure that it is impossible to accidentally compile for a platform/compiler where the memory layout is not exactly the one your code and your external interface require.

Since C11, that is possible using static_assert, sizeof and offsetof.

So something like the following should do the job if you can require C11 (I presume you can require C11 as you are using alignas which is not part of C99 but is part of C11). The "portable C" part here is you fixing the code for each platform/compiler where the compilation fails due to one of the static_assert declarations failing.

#include <assert.h>
#include <stdalign.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>

typedef uint16_t Node8[8];

typedef struct Octree {
    uint64_t *data;
    uint8_t alignas(8) alloc;
    uint8_t dataalloc;
    uint16_t size, datasize, node0;
    Node8 alignas(16) node[];
} Octree;

static_assert(0x10 == sizeof(Octree),              "Octree size error");
static_assert(0x00 == offsetof(Octree, data),      "Octree data position error");
static_assert(0x08 == offsetof(Octree, alloc),     "Octree alloc position error");
static_assert(0x09 == offsetof(Octree, dataalloc), "Octree dataalloc position error");
static_assert(0x0a == offsetof(Octree, size),      "Octree size position error");
static_assert(0x0c == offsetof(Octree, datasize),  "Octree datasize position error");
static_assert(0x0e == offsetof(Octree, node0),     "Octree node0 position error");
static_assert(0x10 == offsetof(Octree, node),      "Octree node[] position error");

The series of static_assert declarations could be written more concisely with less redundant source code typing for the error messages using a preprocessor macro stringifying the struct name, member name, and maybe size/offset value.

Now that we have nailed down the struct member sizes and offsets within the struct, two aspects still need checks.

  • The integer endianness your code expects is the same endianness your memory structure contains. If the endianness happens to be "native", you have nothing to check for or to handle conversions. If the endianness is "big endian" or "little endian", you need to add some checks and/or do conversions.

  • As noted in the comments to the question, you will need to verify separately that the undefined behaviour &(((uint16_t *)octree.node)[-1]) == &octree.node0 actually is what you expect it to be on this compiler/platform.

    Ideally, you would find a way to write this as a separate static_assert declaration. However, such a test is quick and short enough that you can add such a check to the runtime code in a rarely but guaranteed to be run function like a global initialization function, a library initialization functions, or maybe even a constructor. Do be cautious though if you use the assert() macro for that check, as that runtime check will turn into a no-op if the NDEBUG macro is defined.

ndim
  • 35,870
  • 12
  • 47
  • 57
  • I think the syntax is `_Static_assert`, not `static_assert`. But I don't use much C, so can't say for sure. – anatolyg Jun 11 '23 at 14:51
  • 1
    ISO C11 says `assert.h` defines the `assert` and `static_assert` macros, and that the `static_assert` expands to the new-in-C11 keyword `_Static_assert`. – ndim Jun 11 '23 at 15:31