55

I've been using std::memcpy to circumvent strict aliasing for a long time.

For example, inspecting a float, like this:

float f = ...;
uint32_t i;
static_assert(sizeof(f)==sizeof(i));
std::memcpy(&i, &f, sizeof(i));
// use i to extract f's sign, exponent & significand

However, this time, I've checked the standard, I haven't found anything that validates this. All I found is this:

For any object (other than a potentially-overlapping subobject) of trivially copyable type T, whether or not the object holds a valid value of type T, the underlying bytes ([intro.memory]) making up the object can be copied into an array of char, unsigned char, or std​::​byte ([cstddef.syn]).40 If the content of that array is copied back into the object, the object shall subsequently hold its original value. [ Example:

#define N sizeof(T)
char buf[N];
T obj;                          // obj initialized to its original value
std::memcpy(buf, &obj, N);      // between these two calls to std​::​memcpy, obj might be modified
std::memcpy(&obj, buf, N);      // at this point, each subobject of obj of scalar type holds its original value

— end example ]

and this:

For any trivially copyable type T, if two pointers to T point to distinct T objects obj1 and obj2, where neither obj1 nor obj2 is a potentially-overlapping subobject, if the underlying bytes ([intro.memory]) making up obj1 are copied into obj2,41 obj2 shall subsequently hold the same value as obj1. [ Example:

T* t1p;
T* t2p;
// provided that t2p points to an initialized object ...
std::memcpy(t1p, t2p, sizeof(T));
// at this point, every subobject of trivially copyable type in *t1p contains
// the same value as the corresponding subobject in *t2p

— end example ]

So, std::memcpying a float to/from char[] is allowed, and std::memcpying between the same trivial types is allowed too.

Is my first example (and the linked answer) well defined? Or the correct way to inspect a float is to std::memcpy it into a unsigned char[] buffer, and using shifts and ors to build a uint32_t from it?


Note: looking at std::memcpy's guarantees may not answer this question. As far as I know, I could replace std::memcpy with a simple byte-copy loop, and the question will be the same.

geza
  • 28,403
  • 6
  • 61
  • 135
  • As long as they have the same size there shouldn't be a problem. However, if you just need to interpret `f` as `uint32_t` you may just write `(uint32_t&)f`. It will interpret the memory location of the `float` as if it was `uint32_t`. – Piotr Siupa Jul 12 '18 at 08:27
  • 10
    @NO_NAME My experiment shows that your suggestion violates the strict aliasing rules. http://coliru.stacked-crooked.com/a/bb54317049f5c8fc –  Jul 12 '18 at 08:35
  • @NickyC I suppose that `int i = (uint32_t&)f;` wouldn't break the rules. – Piotr Siupa Jul 12 '18 at 08:43
  • I want to note that the [`bit_cast` proposal](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0476r1.html)'s reference [implementation](https://github.com/jfbastien/bit_cast/blob/master/bit_cast.h) uses `memcpy` and `aligned_storage`. –  Jul 12 '18 at 08:48
  • Your approach is at least practically correct. This is how boost handles floating point bits https://www.boost.org/doc/libs/1_45_0/boost/math/special_functions/detail/fp_traits.hpp – llllllllll Jul 12 '18 at 08:48
  • @Bathsheba I cannot imagine safer way to use of this syntax. Are you telling me that part of C++ syntax is invalid by design and shouldn't be used at all? – Piotr Siupa Jul 12 '18 at 08:49
  • 1
    Related: https://stackoverflow.com/questions/3275353/c-aliasing-rules-and-memcpy – xskxzr Jul 12 '18 at 08:50
  • @NickyC: a compiler supplied library code can use anything, because they know how the compiler works, which code with UB can be used. For example, the implementation of `std::vector` always contained UB (I don't know the current state though, maybe with the intruduction of `std::launder`, it is not UB any more). – geza Jul 12 '18 at 08:52
  • @all Sorry, I was answering a different question. I'll get me coat. – Bathsheba Jul 12 '18 at 08:56
  • 1
    Related: https://stackoverflow.com/questions/17789928/whats-a-proper-way-of-type-punning-a-float-to-an-int-and-vice-versa – plasmacel Jul 12 '18 at 09:00
  • 1
    @NO_NAME It still violates the strict aliasing rules. Valid syntax does not imply valid operation. Just like the English sentence ["Colorless green ideas sleep furiously"](https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_furiously) is grammatically correct but meaningless. –  Jul 12 '18 at 09:01
  • @NickyC correct that [definitely breaks the strict aliasing rules](https://stackoverflow.com/a/51228315/1708801) – Shafik Yaghmour Jul 12 '18 at 13:12
  • Have you considered using the functions for extracting exponent and significand rather than reinventing the (apparently UB) wheel? See `frexp()`, [for instance](https://en.cppreference.com/w/cpp/numeric/math/frexp) and perhaps other floating-point manipulation functions in . – Eric Towers Jul 12 '18 at 22:10
  • @EricTowers: Thanks for the suggestion. It can be a solution in some cases, yes. But if I want to have the significand as an integer, it is no use. Furthermore, this float->int was just an example. – geza Jul 13 '18 at 05:34

3 Answers3

25

The standard may fail to say properly that this is allowed, but it's almost certainly supposed to be, and to the best of my knowledge, all implementations will treat this as defined behaviour.

In order to facilitate the copying into an actual char[N] object, the bytes making up the f object can be accessed as if they were a char[N]. This part, I believe, is not in dispute.

Bytes from a char[N] that represent a uint32_t value may be copied into an uint32_t object. This part, I believe, is also not in dispute.

Equally undisputed, I believe, is that e.g. fwrite may have written the bytes in one run of the program, and fread may have read them back in another run, or even another program entirely.

Because of that last part, I believe it does not matter where the bytes came from, as long as they form a valid representation of some uint32_t object. You could have cycled through all float values, using memcmp on each until you got the representation you wanted, that you knew would be identical to that of the uint32_t value you're interpreting it as. You could even have done that in another program, a program that the compiler has never seen. That would have been valid.

If from the implementation's perspective, your code is indistinguishable from unambiguously valid code, your code must be seen as valid.

  • Separating the single steps involved and qualifying each one as undisputed is making things clear. – Peter - Reinstate Monica Jul 12 '18 at 09:25
  • 3
    What the OP observed is interesting though: While the standard in 6.9.2 explicitly permits copying bytes *out of* a a trivially copyable object it lacks (or appears to lack -- I just looked at all occurrences of `memcpy` in n4659) an explicit rule allowing copying bytes *into* such an object. It is probably considered self-understood; the example in 6.9.2 itself copies the bytes back, after all. – Peter - Reinstate Monica Jul 12 '18 at 09:42
  • 5
    @PeterA.Schneider Right. There is "If the content of that array is copied back into the object, the object shall subsequently hold its original value." which grants permission to copy *back into* a trivially copyable object, but general permission to copy into (rather than back into) a trivially copyable object is never explicitly given in the standard, it can only be inferred. That's the gist of my answer. –  Jul 12 '18 at 10:39
  • Good reasoning! I've got a question though. A `float` -> `char[]` copy is OK. A `char[]` -> `uint32_t` is OK too. But, is a direct `float` -> `uint32_t` OK too? – geza Jul 12 '18 at 12:17
  • @geza It's iffy, but I'd say that since treating the bytes in that `float` as a `char[]` is allowed, when you do a direct `float` -> `uint32_t`, in a sense, you *are* copying from a `char[]` to a `uint32_t`. –  Jul 12 '18 at 12:26
  • I believe this to be the best possible answer to this question. – Lightness Races in Orbit Jul 12 '18 at 14:03
  • Interesting... this could also serve as a potential solution to the issue of [enumeration types not being officially layout-compatible with their underlying type](https://stackoverflow.com/q/21956017/5386374), among other things. – Justin Time - Reinstate Monica Sep 26 '19 at 02:16
19

Is my first example (and the linked answer) well defined?

The behaviour isn't undefined (unless the target type has trap representations that aren't shared by the source type), but the resulting value of the integer is implementation defined. Standard makes no guarantees about how floating point numbers are represented, so there is no way to extract mantissa etc from the integer in portable way - that said, limiting yourself to IEEE 754 using systems doesn't limit you much these days.

Problems for portability:

  • IEEE 754 is not guaranteed by C++
  • Byte endianness of float is not guaranteed to match integer endianness.
  • (Systems with trap representations).

You can use std::numeric_limits::is_iec559 to verify whether your assumption about representation is correct.

Although, it appears that uint32_t can't have traps (see comments) so you needn't be concerned. By using uint32_t, you've already ruled out portability to esoteric systems - standard conforming systems are not require to define that alias.

Toby Speight
  • 27,591
  • 48
  • 66
  • 103
eerorika
  • 232,697
  • 12
  • 197
  • 326
  • The unsigned integer types cannot have trap representations, can they? I vaguely recall all of their bits must be value bits. – StoryTeller - Unslander Monica Jul 12 '18 at 09:17
  • 1
    @StoryTeller as far as I know, `unsigned char` is the only type guaranteed to not have traps. – eerorika Jul 12 '18 at 09:33
  • Ah, [it's even explicitly stated](https://timsong-cpp.github.io/cppwp/basic.types#basic.fundamental-1.sentence-9). Interesting, what I recalled was another Q&A (about C, though) that surmised the `uintN_t` types cannot have trap representation. I'll see if I can find it. – StoryTeller - Unslander Monica Jul 12 '18 at 09:43
  • Found the relevant part in the C standard, anyway, http://port70.net/~nsz/c/c11/n1570.html#7.20.1.1 . Those types do not have any padding bits by definition. So all of their bits must be value bits. No possibility of traps in C, anyway. I wonder if the sentence in the C++ standard is an editorial oversight or there's something deeper... – StoryTeller - Unslander Monica Jul 12 '18 at 09:48
  • 2
    @StoryTeller right, so the exact width types aliases have guarantees (beyond tose for the types that they are an alias for, if they alias the standard `int` or such). That's kinda odd, but it's always nice to be able to ignore traps, so I'll take it :) – eerorika Jul 12 '18 at 09:53
  • 2
    "The behaviour isn't undefined". Why? Is there anything in the standard which makes this defined? – geza Jul 12 '18 at 10:06
  • @geza simply because nothing in the standard makes it undefined. – eerorika Jul 12 '18 at 10:14
  • I think if something is not defined by the standard, then it's undefined. Sure, there could be cases, where you can find out some logical working, but it is not enough. For example (as far as I know), the standard doesn't specify what happens for pointer arithmetic, if the pointer doesn't point to an array. So we treat this as UB. Sure, we can find out some logical working for this, but it is still UB. – geza Jul 12 '18 at 10:33
  • @geza The (C) standard defines how `memcpy` behaves, so it is not "not defined". – eerorika Jul 12 '18 at 10:38
  • Sure, it copies bytes. That's all. But we have objects here, not byte arrays. And we have two cases highlighted by the standard. It must mean something, for example it could mean that other use cases are not defined. – geza Jul 12 '18 at 10:42
  • The standard doesn't explicitly state behaviour for every single expression. There are cases where the standard lists possible situations and their behaviour, and if some situation is not mentioned in that rule, then that situation is undefined indeed. But can you name a rule which lists behaviours of `memcpy` in certain situations where your situation isn't included? – eerorika Jul 12 '18 at 10:42
  • Hmm, I think that the standard specifies everything. No wonder it is insanely huge. Name a rule? It is in my question. Two cases of `memcpy` are listed. My use case is not among them. – geza Jul 12 '18 at 10:45
  • @geza Notice that in both rules, `memcpy` is mentioned only in the *example*. Examples are not normative, and just because there isn't an example for something, doesn't mean that it isn't defined. Neither of those rules explicitly define how `memcpy` behaves. Rather, they exemplify the standard rules in terms of `memcpy`. – eerorika Jul 12 '18 at 10:51
  • 2
    Yes, they give an example what they have previously written in the normative text. I could copy bytes with a simple byte-copy loop, the outcome will be the same. So the point is not `memcpy` here, but the principle. Can I copy a `float` to `uint32_t` byte-by-byte by an means? Is it defined by the standard? – geza Jul 12 '18 at 10:55
  • @StoryTeller: uint32_t can have traps, but I know of zero architectures for which a uint32_t that you just took an address of that can still trap on the very next read of it. In fact I only know how to get it to trap if it's still never assigned to. – Joshua Jul 12 '18 at 16:06
  • @Joshua - I find it hard to imagine where one can squeeze in a trap representation when the standard requires all the object's bits are value bits. – StoryTeller - Unslander Monica Jul 12 '18 at 16:09
  • { uint32_t value; value = value + 1; } can trap anyway. But this can only trap on the pointer: { char *ptr = something; uint32_t value = *(uint32_t *)pointer; } – Joshua Jul 12 '18 at 16:11
  • @Joshua - For the addition to trap. The indeterminate value has to be a trap representation to begin with. The C standard implies those don't exist. – StoryTeller - Unslander Monica Jul 12 '18 at 16:21
  • @StoryTeller: Beware. The Itanium processor actually has this behavior. Uninitialized integers can trap. – Joshua Jul 12 '18 at 16:23
  • @Joshua - Then it will likely *not* have uint32_t defined. – StoryTeller - Unslander Monica Jul 12 '18 at 16:25
  • @StoryTeller: But it does. The Itanium has the general property of *any* uninitialized integer or pointer in a local variable can trap. – Joshua Jul 12 '18 at 16:27
  • @Joshua - *sigh* The abstract machine defined by C standard requires *no* trap values for the unsigned fixed width integer types. If they exist on an implementation, they must not trap. However it's accomplished is immaterial. To do otherwise is to not be standard compliant. That requirement is in the link I posted in a previous comment. – StoryTeller - Unslander Monica Jul 12 '18 at 16:34
  • @Joshua As I understand, the Itanium trap bit is "not a thing" that lives only in the register. I don't think such trap can affect a well defined C++ (or C) program - I mean, you cannot produce a value with the trap bit set wrong, since the bit is not part of the value bits. As long as the C++ program has defined behaviour according to the standard, the implementation is required to set whatever trap bit there may exist to their correct value in order to avoid UB. – eerorika Jul 12 '18 at 16:45
  • Now, if my answer is wrong, and this has UB accroding to C++, then the implementation can leave the bit to whatever value having whatever behaviour the processor will have. – eerorika Jul 12 '18 at 16:45
  • @user2079303: You understand correctly. Notice how the only way I could find to reach it was the uninitialized local variable, which might well have been allocated a register. – Joshua Jul 12 '18 at 17:29
  • 2
    Many implementations represent certain data types differently in registers and in memory. It is common on 32-bit RISC platforms, for example, that a `uint16_t` which is placed in a register will have 16 data bits and 16 padding bits which are zeroed when the value is written. If such an object is read when it is uninitialized, it may yield a value which sometimes behaves like it's outside the range 0-65535 but sometimes behaves like it's within that range. Such behavior goes back decades before Itanium. – supercat Jul 12 '18 at 20:24
  • @supercat: can you please elaborate more on this issue with RISC? I don't understand the problem fully here (or just name the CPU/platform please, and I'll look into it). – geza Jul 17 '18 at 20:05
  • @geza: Many 32-bit RISC platforms don't have any 16-bit registers, nor even any 16-bit instructions other than "load or store halfword". Thus, if one declares `uint16_t x` and a compiler decides it will fit in a register, the compiler will use a 32-bit register for it (there is no other kind). Any time the compiler stores a value to `x`, it will ensure the upper bits are all zeroes (masking them off if necessary). If `x` is never written at all, however, the upper bits might hold arbitrary values left over from whatever other purpose that register was used for. – supercat Jul 17 '18 at 20:14
  • @supercat: okay, now I understand this, thanks :) What can go wrong for my float->int example on such a platform (I mean, besides the usual thing that I'll get garbage result)? – geza Jul 17 '18 at 20:26
  • @geza: I was responding more generally to the comments about uninitialized variables. I should mention, though, looking back through your comments, that despite its increasing size, I doubt that the C++ Standard completely describes the behavior of a useful quality C++ implementation that's suitable for any particular purpose, any more than the C Standard completely describes the behavior of a quality C implementation that's likewise suitable. – supercat Jul 17 '18 at 20:38
  • @geza: As for what can go wrong, I don't think anything should go wrong when targeting an implementation that's suitable for low-level programming on any remotely-common platform. Some "clever" compiler writers, however, are more interested in exploiting places where the Standard would let them behave nonsensically, for purposes of "optimization", then in exploiting ways in which they could easily and cheaply be made to behave usefully. – supercat Jul 17 '18 at 20:42
  • @supercat: yes, of course, current C++ implementations are useful. They would be even useful, if the C++ standard would be half of its current size. But a standard should specify everything, that's why it's a standard. If it doesn't specify something, then it is unspecified, i.e. unspecified behavior. We can "specify" these by ourselves in a logical manner, but it would not be backed up by the standard. And this thinking actually caused problems in the past. UBs which previously didn't cause problems, with a new compiler version, do cause problems, etc. – geza Jul 18 '18 at 06:19
  • @geza: The best the Standard could hope to do--and what I think it should do--is define a category of Safely Conforming implementations and Selectively Conforming programs such that feeding any Selectively Conforming program to any Safely Conforming implementation will cause it to behave it in defined fashion, indicate via Implementation-Defined means a refusal to do so, or hang (which would be equivalent to executing the code at infinitesimally speed). Then add enough optional features to maximize the range of tasks that can be done with Selectively Conforming programs... – supercat Jul 18 '18 at 06:33
  • ...without having to care about whether all implementations can support all the features defined by the Standard. The range of programs an implementation could usefully process would be a Quality of Implementation issue, but jumping the rails when hitting a translation limit (as opposed to terminating execution in an Implementation-Defined manner) would make an implementation non-compliant. Unfortunately, the Standard fails to define the behavior of most useful programs and relies upon implementations to use at least some common sense. – supercat Jul 18 '18 at 06:39
15

Your example is well-defined and does not break strict aliasing. std::memcpy clearly states:

Copies count bytes from the object pointed to by src to the object pointed to by dest. Both objects are reinterpreted as arrays of unsigned char.

The standard allows aliasing any type through a (signed/unsigned) char* or std::byte and thus your example doesn't exhibit UB. If the resulting integer is of any value is another question though.


use i to extract f's sign, exponent & significand

This however, is not guaranteed by the standard as the value of a float is implementation-defined (in the case of IEEE 754 it will work though).

Hatted Rooster
  • 35,759
  • 6
  • 62
  • 122
  • If we can deduce this from the description of `memcpy`, then why does the standard highlight the two cases I mentioned in my question? – geza Jul 12 '18 at 10:04
  • @geza C++ will rely on C for the definition of memcpy and as I said [here](https://stackoverflow.com/questions/98650/what-is-the-strict-aliasing-rule/51228315#comment89454229_51228315) it says it copies n bytes "w/o condition" between two objects. – Shafik Yaghmour Jul 12 '18 at 13:08
  • Note [bit_cast](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0476r2.html) uses memcpy as the underlying type punning mechanism. – Shafik Yaghmour Jul 12 '18 at 13:11
  • @ShafikYaghmour: yes, but the problem is not there. For example, if you `memcpy` a non-trivial copyable type, it is UB, because the standard doesn't allow it. Like it doesn't allow my example explicitly. – geza Jul 12 '18 at 13:15
  • @ShafikYaghmour: it's not a problem. A compiler provided library can use any UB it wants, see my comment below the question. – geza Jul 12 '18 at 13:15
  • @geza like I said, you are allowed to do it by definiton of memcpy, whether you obtain a valid representation is a different and separate story. They don't have to be linked together. It could be stated more explicitly though. – Shafik Yaghmour Jul 12 '18 at 13:19
  • @ShafikYaghmour: In my opinion, this has nothing to do with the guarantees of `memcpy`. The question would be exactly the same, if I use a simple byte-copy loop instead of `memcpy`. See the comments below user2079303's answer. – geza Jul 12 '18 at 13:30
  • The standard allows aliasing some object as a `char*`, yes, but that says nothing about implicitly rematerialising it as some other object type via said aliasing. (I do think this is legal - it's the conventional alternative to the broken `reinterpret_cast(&src)` approach - but I don't think anyone here's proven it yet!) – Lightness Races in Orbit Jul 12 '18 at 14:02
  • @LightnessRacesinOrbit I think the fact that the standard allows reading from and writing to a `char*` that's aliasing some other object it implies that this allows `memcpy` to rematerialise by reading from `src` and writing to `dest`. Essentially copying the underlying bytes. Although I agree this might be room to interpretation. – Hatted Rooster Jul 12 '18 at 14:46
  • @SombreroChicken: Yep I agree with hvd that it's almost certainly intended to work for trivially-copyable types but that it's unfortunately underspecified. – Lightness Races in Orbit Jul 12 '18 at 15:09
  • @geza "_For example, if you memcpy a non-trivial copyable type, it is UB, because the standard doesn't allow it_" I don't think so: all objects are made up of bytes. Reading those bytes cannot be disallowed. (It's also pointless as putting those bytes elsewhere isn't guaranteed to produce an object with a well defined state.) – curiousguy Jul 12 '18 at 23:36
  • @curiousguy: https://stackoverflow.com/questions/27009178/when-is-a-type-in-c11-allowed-to-be-memcpyed – geza Jul 13 '18 at 05:24