PACKING data without using the "PACKED" compiler directive in C

Question

Background: It's a 16 bit TI DSP (TMS320F2812 to be precise). the DSP is little-endian. The compiler is C2000 (which does NOT support PACKED directive). I need to communicate several structures of varied sizes from source to destination through the ethernet. The problem being the protocol requires PACKED data (padded bytes will also be considered as information).

What I currently plan to do: Use Bitfields in the structures

typedef struct
{
   INT8U channel:4;
   INT8U priority:4;
   INT16U length;
   INT16U address;
   INT8U array[4];
} _MBXh;

Doubt: In this particular structure, the "INT16U length" will start from a new aligned memory address (please correct me if my understanding is wrong). Hence there will be a padding of (16 bits - (4+4)bits =) 8 bits after the "INT8U priority".

Q1 -> Does this happen even with compiler directive "pack"? (ofcourse it depends on compiler but my question is related to the c99 standard on which I could not find information). What I found was c99 provides tight packing. But with that definition, I don't understand if the "INT16U length" will start immediately after "INT8U priority" or after 8 bit padding.

Q2 -> Arrays inside structures cannot be assigned bitfields. If there are 8 bit elements inside the array then every element of the array will padded with 8 more bits to align to the 16 bit processor.

Another alternative solution is to use char* pointer to point to the struct at the time of transmission (or reception).

Q3 -> In this case I need to manually "combine" INT8U channel and INT8U priority. Which will become difficult if there are many structures declared. Please correct me if this understanding is wrong.

Q4 -> Please help me with a more elegant solution to the problem. I need to pack data (including bitfields and ARRAYS inside structures) but I do not have the compiler directive.

Answer 1

As unwind described in a comment, you should instead serialize (on write) and deserialize (on read) the structure comments to/from a byte buffer.

There are a number of ways to do this. For example, inlined functions (C99 static inline), preprocessor macros, individual functions per each field, a generic function to bit-pack fields, and so on.

The most common option is to pack and unpack byte arrays from/to internal structures. For example, for the internally-used structure

struct mbxh {
    INT8U channel:4;
    INT8U priority:4;
    INT16U length;
    INT16U address;
    INT8U array[4];
};

static void pack_mbxh(unsigned char *const dst, const struct mbxh *src)
{
    dst[0] = src->channel | ((src->priority) << 4);
    dst[1] = src->length >> 8;
    dst[2] = src->length;
    dst[3] = src->address >> 8;
    dst[4] = src->address;
    dst[5] = src->array[0];
    dst[6] = src->array[1];
    dst[7] = src->array[2];
    dst[8] = src->array[3];
}

static void unpack_mbxh(struct mbxh *dst, const unsigned char *const src)
{
    dst->channel = src[0] & 15U;
    dst->priority = (src[0] >> 4) & 15U;
    dst->length = (src[1] << 8) | src[2];
    dst->address = (src[3] << 8) | src[4];
    dst->array[0] = src[5];
    dst->array[1] = src[6];
    dst->array[2] = src[7];
    dst->array[3] = src[8];
}

This is especially useful, because it makes it trivial to specify the byte order; the above uses big-endian or network byte order for the length and address fields.

If the target system is very RAM-constrained, using preprocessor macros to access the "packed" fields directly, is often a good option. This uses less memory, but more CPU resources. (Note that the "packed" fields use big-endian or network byte order here, too.)

#define mbxh_get_channel(data)  ((data)[0] & 15U)
#define mbxh_get_priority(data) ((data)[0] >> 4)
#define mbxh_get_length(data)   ((((INT16U)(data)[1]) << 8) | ((INT16U)(data)[2]))
#define mbxh_get_address(data)  ((((INT16U)(data)[3]) << 8) | ((INT16U)(data)[4]))
#define mbxh_get_array(data, i) ((data)[i])

#define mbxh_set_channel(data, value)                                 \
        do {                                                          \
            (data)[0] = ((data)[0] & 240U) | ((INT8U)(value)) & 15U); \
        } while (0)

#define mbxh_set_priority(data, value) \
        do {                           \
            (data)[0] = ((data)[0] & 15U) | (((INT8U)(value)) & 15U) << 4); \
        } while (0)

#define mbxh_set_length(data, value)            \
        do {                                    \
            (data)[1] = ((INT16U)(value)) >> 8; \
            (data)[2] = (INT8U)(value);         \
        } while (0)

#define mbxh_set_address(data, value)           \
        do {                                    \
            (data)[3] = ((INT16U)(value)) >> 8; \
            (data)[4] = (INT8U)(value);         \
        } while (0)

#define mbxh_set_array(data, index, value)    \
        do {                                  \
            (data)[(index)] = (INT8U)(value); \
        } while (0)

In practice, especially if you have many such structures, a combination of these will work. First, you write some compact functions to access each type of field: a low nibble, a high nibble, or a 16-bit field,

static INT8U get4u_lo(const INT8U *const ptr)
{
    return (*ptr) & 15U;
}

static INT8U get4u_hi(const INT8U *const ptr)
{
    return (*ptr) >> 4;
}

static INT16U get16u(const INT8U *const ptr)
{
    return (((INT16U)ptr[0]) << 8) | ptr[1];
}

static void set4u_lo(INT8U *const ptr, INT8U val)
{
    *ptr &= 240U;
    *ptr |= val & 15U;
}

static void set4u_hi(INT8U *const ptr, INT8U val)
{
    *ptr &= 15U;
    *ptr |= (val % 15U) << 4;
}

static void set16u(INT8U *const ptr, INT16U val)
{
    ptr[0] = val >> 8;
    ptr[1] = val;
}

Next, you write the per-structure field accessors using the above helper functions:

#define mbxh_get_channel(data)  get4u_lo((INT8U *)(data)+0) 
#define mbxh_get_priority(data) get4u_hi((INT8U *)(data)+0)
#define mbxh_get_length(data)   get16u((INT8U *)(data)+1)
#define mbxh_get_address(data)  get16u((INT8U *)(data)+3)
#define mbxh_get_array(data, i) ((data)[5+(i)])

#define mbxh_set_channel(data, v)  set4u_lo((INT8U *)(data)+0, (v))
#define mbxh_set_priority(data, v) set4u_hi((INT8U *)(data)+0, (v))
#define mbxh_set_length(data, v)   set16u((INT8U *)(data)+1, (v))
#define mbxh_set_address(data, v)  set16u((INT8U *)(data)+3, (v))
#define mbxh_set_array(data, i, v) ((data)[5+(i)] = (v))

As in all of the examples in this answer, the above too use big-endian or network byte order for the data. channel is in the four low bits, and priority in the four high bits, of the first data byte.

Overall, I recommend the first option (structure conversion per function call) for desktop applications, and for cases where you use an internal structure anyway. For microcontrollers and other constrained-memory cases, I recommend this latest one.

(None of the above code is tested. If you find typos or bugs or other errors, please notify me in a comment, so I can fix the example code above.)

Answer 2

You should serialize your data, for example

static unsigned char * Ser_uint16_t(unsigned char *p, uint16_t value)
{
    *p++ = (value >> 8u) & 0xFFu;
    *p++ = (value >> 0u) & 0xFFu;

    return p;
}

void serialize( INT8U channel,
                INT8U priority,
                INT16U length,
                INT16U address,
                INT8U array[4]
                unsigned char *out_buffer)
{
    out_buffer++ = ((channel & 0x0F) << 4) | (priority & 0x0F);
    out_buffer = Ser_uint16_t(out_buffer, length);
    out_buffer = Ser_uint16_t(out_buffer, address);
    memcpy(out_buffer, array, 4);
}

and de-serialize them on the other side, like

static uint16_t DeSer_uint16_t(unsigned char **p)
{
    uint16_t x = 0;

    x |= ((uint16_t)(*p)[0]) <<  8u;
    x |= ((uint16_t)(*p)[1]) <<  0u;

    *p += 2;

    return x;
}