2

I'm trying to build a simple custom application layer protocol, essentially carrying a timestamp, plus some other useful information, in order to perform few network measurements between different Linux systems.

The implementation, in my initial idea, should be as portable as possible between different platforms (x86, ARM, ...), as far as Linux systems are concerned.

In order to manage the header, I've created this structure:

struct myhdr {
    __u8 reserved; // 1 byte
    __u8 ctrl; // 1 byte
    __u16 id; // 2 bytes
    __u16 seq; // 2 bytes
    __u16 len; // 2 bytes
    struct timeval sendtime; // 8 or 16 bytes
};

After which some payload data may or may not (if len=0) be present. Since this data has to be sent over the network, I need, if I'm not wrong, the structure to be packed, without any alignment padding.

My doubt is actually whether this can be considered as already packed or not, mainly due to the presence of struct timeval, to carry the timestamp.

Under 32 bit systems, struct timeval should be 8 bytes. Under 64 bit systems, it should be 16 bytes (just tested this by printing sizeof(struct timeval)).

Under 32 bit systems, is it safe to assume it to be already packed, due to the fact that 1+1+2+2+2 bytes = 8 bytes, which is the size of sendtime? Or will be a padding added in any case in order to align every single field to the last one, which is the largest?

What happens then in 64 bit systems, where the last field is 16 bytes? I think that the structure won't be "packed by layout" anymore, in any case (is this correct?).

Is adding __attribute__((packed)) sufficient and always necessary to ensure that the structure is packed when the code is compiled for different platforms? Are there better solutions?

alk
  • 69,737
  • 10
  • 105
  • 255
es483
  • 361
  • 2
  • 16
  • 3
    There's more problems than packing you need to think about, most notable [*endianness*](https://en.wikipedia.org/wiki/Endianness). That, and the structure padding problem, is why raw binary data is so uncommon these days. If transferring binary data, consider some form of [*serialization*](https://en.wikipedia.org/wiki/Serialization) so you don't have to worry about it. – Some programmer dude Jan 13 '19 at 17:03
  • `__attribute__((packed))` is *never necessary*, it should be avoided like plague. – Antti Haapala -- Слава Україні Jan 13 '19 at 19:23

1 Answers1

5

If you define a wire protocol, you really need to use your own type. To be on the safe side, you should use 64 bits for the seconds from 1970, even if many 32-bit systems still use a 32-bit counter.

struct timeval comes from a system header and can have basically any size, starting with 8 bytes. I'm sure there are 32-bit systems out there where it has size 12 (64-bit time_t for tv_sec to avoid the Y2038 problem, 32-bit long/suseconds_t for tv_usec). Furthermore, POSIX only requires that struct timeval has certain members. Some systems have explicit fields in their system headers to avoid implicit padding by the compiler, leading to further differences in (hypothetical) packing behavior.

And while __attribute__ ((pack)) does not apply recursively (so sizeof (p->sendtime) will equal sizeof (struct timeval), it still reduces alignment of the entire struct, including the the sendtime member, to 1, so the member can be misaligned and unsuitable for direct use with functions which expected a struct timeval *.

Florian Weimer
  • 32,022
  • 3
  • 48
  • 92
  • Thank you very much for your reply! I will define, instead of ´struct timeval´, two 64 bit fields, in order to be safe both on 32 bit and 64 bit systems. Would this make my structure already "packed by layout", since the first fields sum up to 8 bytes, followed by two 8 bytes fields (so that I can avoid ´__attribute__((packed))´ as much as possible, as suggested in another comment)? – es483 Jan 15 '19 at 09:55
  • 1
    Yes, with this approach, you only need to make sure that the data is properly aligned (`memcpy` from a transfer buffer can do this; the compiler will typically eliminate that) and deal with endianness differences. – Florian Weimer Jan 15 '19 at 19:15