Packed structures and not packed without code repetition

Question

I have this code:

// size probably 4 bytes
struct NotPacked
  {
  uint16_t first;
  uint8_t second;
  };

// size is 3 bytes
struct Packed
  {
  uint16_t first;
  uint8_t second;
  }__attribute__((packed));

I'd like to use the same structure, sometimes as packed, sometimes not. Do you know any way to write this code avoiding repetition?

[Edit] Question should have been: ".. to write this code avoiding as much code duplication as possible"

[Edit2] I've tried an experiment using empty class optimization but no success

[Edit3] Basic example added:

  Packed packet;
  receivePacketFromNetwork(&packet); // Fill the structure with data coming from the network
  NotPacked notPacked = packedToUnpacked(packet); // convert packed structure to unpacked
  processUnpacked(notPacked); // Do a lot of computations

To rephrase the question: how do I use the same code on two different data structures? Or, to quote Fred Picker, "different isn't the same". — Pete Becker, Jun 24 '19 at 14:00
let's say almost different. For instance specifying a template parameter. I'm pretty sure it would work with MACROs but I'd like to avoid that — Saturnu, Jun 24 '19 at 14:03
Can you give us an example on how would you use this two structures? — Silvano Cerza, Jun 24 '19 at 14:08
@Evg mmm for templates maybe I have an idea, I'll try it and edit the question if it works — Saturnu, Jun 24 '19 at 14:16
@SilvanoCerza I'd use the packed one for packets to be sent in the network and the other one instead for internal computations (using memcpy and other helpers) — Saturnu, Jun 24 '19 at 14:17
Code example is the one I wrote :) the real one is much bigger. — Saturnu, Jun 24 '19 at 14:24
@Evg I said it before. Just using these structures as networking packets (to use less space) or for computation (for good performances). Nothing in particular. There's no need and space to write all the networking code here — Saturnu, Jun 24 '19 at 14:50
Nobody is asking for networking code, just a minimal example. Without any code it is not clear what the question is. What does "use the same structure" mean? You want a function that accepts both of them? Use templates. You want to convert one structure into another implicitly? You can define implicit constructors and type conversions operators. Or you just want to *define* two structures without code repetition? — Evg, Jun 24 '19 at 15:12
Ok I'll edit the question with a minimal example. As "use the same structure" I mean just: "using the same fields without writing them two times". Writing template functions that work with these fields should work for both versions. — Saturnu, Jun 24 '19 at 15:25

score 2 · Answer 1 · answered Jun 24 '19 at 14:02

2

I'd like to use the same structure, sometimes as packed, sometimes not. Do you know any way to write this

There isn't.

How on earth do you expect to have two totally different memory layouts defined by the same code?

answered Jun 24 '19 at 14:02

Paul Evans

27,315
3
37
54

As I wrote above, I was wrong in my question. I'd say: "avoiding a lot of code duplication, the more you can". Using Macros should work. I was hoping some trick using templates or "using Foo = blabla " – Saturnu Jun 24 '19 at 14:05
1

MACROs are evil and should be avoided at all cost. Why not simply a `Packed` and `NotPacked` as in your question? That way the code is clear and explicit. – Paul Evans Jun 24 '19 at 14:07
I'd prefer avoid macros, but to avoid at all cost is also code duplication :) – Saturnu Jun 24 '19 at 14:14
what If the structure Packed and not packed is 400 lines long ? if you need to add a field in one of the structures you (or someone else) might forget to add in the other – Saturnu Jun 24 '19 at 14:15
1

It's not "two totally different memory layouts" if there's a common prefix, as there is in the shown example – Lightness Races in Orbit Jun 24 '19 at 14:57
@Saturnu then you have to asked yourself what's the purpose of the unpacked one? – Paul Evans Jun 24 '19 at 15:01
@LightnessRacesinOrbit ok, fair point. But why name two differing abstractions the same? – Paul Evans Jun 24 '19 at 15:03
@PaulEvans I said it before. Unpacked structures are faster than packed ones, so I need them for computation purposes. I can name them slightly different, ideally I'd create a templated struct with two specializations: one for the packed one, another for the non packed one, both using a common helper structure – Saturnu Jun 24 '19 at 15:07
@Saturnu Then it seems like you want `Packer` and `Unpacker` helpers that take `Packed` and `Unpacked` objects and transform them. How else are you going to shift the data? – Paul Evans Jun 24 '19 at 15:11
@Saturnu Then you want to have explicit names for all the abstractions involved in this process. Looks like you'll need some pre-processor hackery unfortunately. – Paul Evans Jun 24 '19 at 15:56
Yep, maybe using templates with specializations I can simplify a bit.. I really wanted to avoid macros.. sometimes better macros than code duplication though – Saturnu Jun 24 '19 at 15:58

Michael Veksler · Answer 2 · 2019-06-26T15:00:27.123

The only way I can think of is to use ugly macros:

#define DEFINE_BOTH_STRUCTS(PACKED_NAME, REGULAR_NAME, ...) \
    struct PACKED_NAME { __VA_ARGS__ } __attribute__((packed)); \
    struct REGULAR_NAME { __VA_ARGS__ }

And then the usage

DEFINE_BOTH_STRUCTS(Packed, NotPacked, 
    char a;
    int b;
    char c[3];
);

This will define both variants out of a single code.

There is also a less desirable option:

#define MY_STRUCT  { \ 
    char a; \
    int b; \
    char c[3]; \
}
struct Packed MY_STRUCT __attribute__((packed));
struct NotPacked MY_STRUCT;
#undef MY_STRUCT

It is less desirable since it requires to define one macro for each struct pair, whereas the former defines only one macro for the whole program. Since macros don't have a namespace, and hence can interact badly, it is advisable to minimize their use (if not possible to avoid them altogether).

Edit: As has been pointed out, having undef in the second solution limits the pollution.

Besides, the undef makes it possible to reuse the same macro without interfering with other macro names.

This is still imperfect since some other code may rely on its own independent MY_STRUCT macro, and our use of MY_STRUCT can still break it by inadvertently redefining and, later, undefinining it.

Thank you Michael. Yep, the macro thing is the only option I had in my mind. It works for sure. If there was another way... — Saturnu, Jun 24 '19 at 14:51
Note that the 2nd answer, you can #undef MY_STRUCT after using it, avoiding name pollution. You can also detect if someone had already used MY_STRUCT and error out instead of smashing it. — Yakk - Adam Nevraumont, Jun 26 '19 at 14:46
@Yakk-AdamNevraumont good point. Updated the answer. Still, MY_STRUCT may collide with a macro defined elsewhere, but that can also happen with the first solution — Michael Veksler, Jun 26 '19 at 15:03
@MichaelVeksler First, `#ifdef MY_STRUCT #pragma error("MY_STRUCT already defined, bailing out")` or something — Yakk - Adam Nevraumont, Jun 26 '19 at 15:04

Yakk - Adam Nevraumont · Accepted Answer · 2019-06-25T17:49:37.990

You could use universal member pointers to access it.

First, define your members in a namespace:

namespace universal {
  template<class T, unsigned int idx=0> struct member_ptr; // TODO
  template<auto const*...> struct packed_struct; // TODO
  template<auto const*...> struct unpacked_struct; // TODO
  template<class...Ts> using variant=std::variant<Ts...>;
  template<class...Ts> struct mutable_pointer:std::variant<Ts*...>{/*TODO*/};
  template<class...Ts> using const_pointer = mutable_ptr<Ts const...>;
}

namespace Foo {
  universal::member_ptr<int16_t> first;
  universal::member_ptr<int8_t> second;

  using packed = universal::packed_struct< &first, &second >;
  using unpacked = universal::unpacked_struct< &first, &second >;

  using either = universal::variant<packed, unpacked>;
  using either_cptr = universal::const_pointer<packed, unpacked>;
  using either_mptr = universal::mutable_pointer<packed, unpacked>;
}

then you can do:

void receivePacketFromNetwork( Foo::either_mptr ptr ) {
  assert(ptr);
  ptr->*Foo::first = 7;
  ptr->*Foo::second = 3;
}

and have it work on both types of structure.

Writing the stuff in namespace universal isn't easy, but it isn't impossible.

The basic idea is to overload operator->*.

template<class T>
struct member_ptr {
  template<class...Ts,
    std::enable_if_t< supports<Ts>() && ..., bool> = true
  >
  T& operator->*( std::variant<Ts...>& lhs, member_ptr const& self ) {
    return std::visit(
      [&self]( auto&& lhs )->T&{ return lhs->*self; },
      lhs
    );
  }
  template<class U>
  constexpr static bool supports(); //TODO
};

template<auto const* a, auto const* b, auto const*... bs>
struct unpacked_struct<a, b, bs...>:
  unpacked_struct<a>,
  unpacked_struct<b, bs...>
{
  using unpacked_struct<a>::operator->*;
  using unpacked_struct<b, bs...>::operator->*;
};

template<class T, , unsigned int idx, member_ptr<T, idx> const* a>
struct unpacked_struct<a> {
  T data;
  T& operator->*( member_ptr<T, idx> const& ) & {
    return data;
  }
  T&& operator->*( member_ptr<T, idx> const& ) && {
    return std::move(data);
  }
  T const& operator->*( member_ptr<T, idx> const& ) const& {
    return data;
  }
  T const&& operator->*( member_ptr<T, idx> const& ) const&& {
    return std::move(data);
  }
};

etc.

Really interesting approach. So the Packed structure should have the attribute packed somewhere right? — Saturnu, Jun 26 '19 at 13:47
@Saturnu Or, maybe it is concatenated memory of the size of the fields, and `->*` returns a pseudo-ref that has read-from and write-to operators on it, and read-to and write-from uses memcpy and statically asserts that the data is trivially copyable. (packing is just a suggestion). Note that I'm not advising this approach; my hobby is to find C++ questions whose answer is "it cannot be done" and then answer them, even if the answer is ridiculous. — Yakk - Adam Nevraumont, Jun 26 '19 at 14:43
Ahah I guess however you've found the solution without the macro.. it's a bit complicated but it should work, thanks! PS I was hoping for something more concise but I think that is only possible using macros — Saturnu, Jun 26 '19 at 15:19

Packed structures and not packed without code repetition

3 Answers3