4

Related to, but somewhat different from, Do any compilers transfer effective type through memcpy/memmove

In C89, memcpy and memmove are required to behave as though the source and destination are accessed using character types, copying all the bits of the source to the destination without regard for the type of data being copied.

C99 changes the semantics so that if an object with an effective type is copied to storage which has no declared type [typically storage received from malloc or other such function], it creates an object in destination storage which may only be accessed using the source type.

The following code, for example, would have fully-defined behavior in C89 on all platforms where "unsigned int" and "unsigned long" have the same 32-bit representation, but would have Undefined Behavior under C99.

#include <stdio.h>
#include <string.h>
void increment_32_bit_uint(void *p)
{
  uint32_t temp;
  memcpy(&temp, p, sizeof temp);
  temp++;
  memcpy(p, &temp, sizeof temp);
}
int main(void)
{
  unsigned *ip = malloc(sizeof (unsigned));
  unsigned long *lp = malloc(sizeof (unsigned long));
  *ip = 3; *lp = 6;
  increment_32_bit_uint(ip);
  increment_32_bit_uint(lp);     
  printf("%u %lu", *ip, *lp);
  return 0;
}

Under the C99 rules, passing allocated storage to "increment_32_bit_uint" function will make it set the Effective Type to uint32_t, which cannot be the same type as both "unsigned" and "unsigned long" even if all three types have the same representation. Consequently, a compiler could do anything it likes with code that reads that storage as a type other than uint32_t, even if that type has the same representation.

Is there any way, in C99 or C11, of performing the copy in a way that would allow a compiler to generate efficient code, but would force the compiler to treat the destination as though it contained a pattern of bits with no effective type [which could thus be accessed using any type]?

Community
  • 1
  • 1
supercat
  • 77,689
  • 9
  • 166
  • 211
  • gcc compiles your example with no warning using `-std=c99` or `-std=c11` after including `` and `` – xvan Mar 11 '16 at 23:47
  • 2
    @xvan: That a particular compiler (or even every compiler that exists today) happens to do something does not mean that the Standard imposes any requirement to keep doing so. There are a number of cases where nearly every compiler in existence had identical behavior for decades without the Standard requiring them to do so, until some compiler writers decided they no longer needed to support those cases, so the fact that code works on all of today's compilers does not mean it doesn't invoke UB. – supercat Mar 11 '16 at 23:50
  • 1
    @xvan: Per N1570: "If a value is copied into an object having no declared type using memcpy or memmove, or is copied as an array of character type, then the effective type of the modified object for that access and for subsequent accesses that do not modify the value is the effective type of the object from which the value is copied, if it has one." I see no basis for saying that the effective type would not be set to `uint32_t`, nor that reading as any other type would not invoke Undefined Behavior. – supercat Mar 11 '16 at 23:52
  • 2
    I don't see why you think it's Undefined Behaviour, after `memcopy()` `p` will have an effective type of `uint32_t`. Outside of `increment_32_bit_uint()` scope `p` doesn't exists any more, thus the first line of the paragraph you cited applies: `The effective type of an object for an access to its stored value is the declared type of the object, if any.` – xvan Mar 12 '16 at 00:28
  • @xvan: The object at the location identified at the location to which `p` points would still exist, and it gets read in the `printf` statement. Compilers are allowed to replace functions with their inline expansion, and a compiler that did so could notice that if `int32_t` is `int` that a `memcpy` cannot possibly affect the value of something that will be read using a `long*` (except by invoking UB). – supercat Mar 12 '16 at 15:06
  • I don't understand, the effective type is for acesses that don't modify the stored value. My understanding is that memcopying to a void *, pointing to a long segment, would change it to the effective type of the memcopying source without UB. Further accesses as long should attempt to use long type. If the compiler can't guarantee that access, it'd fall back to char access, not invoke UB. – xvan Mar 12 '16 at 17:58
  • 1
    @xvan: The `memcpy` itself won't invoke UB, but unless the (source) code which performs the `memcpy` knows the type that is next going to be used to access the data written thereby, it can't avoid setting the effective type in such a way that code which tries to access the data using an unexpected type won't invoke UB. If it hadn't become fashionable for compilers to be so aggressive there wouldn't be an issue, but even though the C99 memcpy rules are 99% useless for optimization, I see no reason to believe compiler writers aren't going to start using them when the Standard says they can. – supercat Mar 12 '16 at 18:18
  • Why not simply make the function parameter `unsigned char*`? – Lundin Mar 14 '16 at 08:32
  • On VS2013 C++ Express: I needet to include '' and '' Output was '4 7'. Switch '/Wall', No Errors, 2 Warnings (2 x 'C4711' increment_32... selected for inlining) – Johannes Mar 14 '16 at 10:22
  • @Johannes If this is undefined behavior, then what output you get from any given compiler is irrelevant. I'm not quite sure if it is UB, but in case it is and caused by pointer aliasing, using a character type should solve it. – Lundin Mar 14 '16 at 13:41
  • @Lundin Thanks for pointing that out. I was curious what VS would do do I compiled it and posted it as a comment since it is definitely irrelevant as an answer. – Johannes Mar 14 '16 at 15:13
  • @Lundin: What difference would the parameter type make? What is required is that the memory be written in such a fashion that it may be given to code that expects either a 32-bit `int` or a 32-bit `long`. If `sizeof (int)` is 4 [it could legitimately be 2 if `char` is 16 bits, or 1 if 32], one could write code to manipulate four bytes at the destination independently as bytes, but that would likely be slow. – supercat Mar 14 '16 at 15:14
  • @Lundin: Note that if a compiler doesn't apply some kind of effective-type rules to memcpy it's often going to be a dog from a performance standpoint since a compiler would have to assume that both source and destination could alias any global variable anywhere in the universe; even if compilers don't yet apply the effective-type rules to memcpy, I would think it'd be an effective target for future "improvements". – supercat Mar 14 '16 at 15:18
  • @supercat If the parameter was a char type, then the effective type would be char since that is the declared type. And character types are excluded from aliasing optimizations. – Lundin Mar 14 '16 at 15:20
  • 1
    @Lundin: The effective type rules don't mention the type of the *argument* to memcpy as playing a role--merely the type of the *object* identified thereby. A much more sensible and useful rule would have said that if the source type isn't void* or a character pointer, it must identify the type of the source object, and if the destination type isn't void* or a character pointer, the destination's effective type will be set to match. That's not what the rule actually *says*, however. Actually, better yet would be to add new functions that are like memcpy/memmove, but have the new rules. – supercat Mar 14 '16 at 15:28

1 Answers1

1

You could get rid of all effective type problems if you just use a return type for the function.

uint32_t increment_32_bit_uint (const void* p)
{
  u32_t result;
  memcpy(&result, p, sizeof(result));
  result++;
  return result;
}

This will force the caller to be careful with their types. Though of course, in theory this is some sort of immutable object rather than an in-place change of the variable. In practice though, I think you'll get the most effective code from this anyway, if you use it as

x = increment_32_bit_uint(&x);

Generally, I don't see how any strict aliasing optimizations would ever be useful in real-world applications if they don't treat the stdint.h types as compatible types to their primitive data type equivalents. Particularly, it must treat uint8_t as a character type, or all professional low-level C code would break.

The same for your case here. If the compiler knows that unsigned int is 32 bit, why would it decide to cause aliasing problems for users of uint32_t and vice versa? That's how you turn a compiler useless.

Lundin
  • 195,001
  • 40
  • 254
  • 396
  • Although I used a single data item in this example, bigger problems occur with arrays [e.g. write a function to decrement every item in an array that isn't already zero] so using the function's return value wouldn't work. Also, if you play with the on-line compilers on e.g. gcc.godbolt.org, you'll notice that even though both "int" and "long" are 32 bits, if a function accepts both an `int*` and a `long*` the compiler will assume the two pointers cannot access the same object. It's not uncommon for a program to have some libraries that expect an array of `unsigned`, some that expect... – supercat Mar 14 '16 at 14:50
  • ...an array of `unsigned long`, and some that expect an array of `uint32_t`, and for the program to need to exchange data among those libraries. There are also many cases where programs may need to work with pointers to various kinds of structures that share a common initial sequence; today's compilers don't allow that, however, unless `memcpy` is used, and even when using `memcpy` the effective type rules don't seem to offer any guarantee that it will work. – supercat Mar 14 '16 at 14:54
  • @supercat Then I suppose the only option is to use a union, with type punning to/from a char type. – Lundin Mar 14 '16 at 15:16
  • Using discrete accesses of char type would be legal with or without a union, but would likely be slow. Having a union of an `int` and a `long` as the memcpy source type might work, and I think the authors of the Standard intended that it should work, but nothing specifically says that an object of union type may be accessed using a pointer to a member type. That wouldn't help with another problem scenario though, which perhaps I should have listed in the question, which is a structure of unknown type with a known initial sequence. – supercat Mar 14 '16 at 15:25