Portable tagged pointers

Question

Is there a portable way to implement a tagged pointer in C/C++, like some documented macros that work across platforms and compilers? Or when you tag your pointers you are at your own peril? If such helper functions/macros exist, are they part of any standard or just are available as open source libraries?

Just for those who do not know what tagged pointer is but are interested, it is a way to store some extra data inside a normal pointer, because on most architectures some bits in pointers are always 0 or 1, so you keep your flags/types/hints in those extra bits, and just erase them right before you want to use pointer to dereference some actual value.

const int gc_flag = 1;
const int flag_mask = 7; // aka 0b00000000000111, because on some theoretical CPU under some arbitrary OS compiled with some random compiler and using some particular malloc last three bits are always zero in pointers.

struct value {
   void *data;
};

struct value val;
val.data = &data | gc_flag;
int data = *(int*)(val.data & flag_mask);

https://en.wikipedia.org/wiki/Pointer_tagging

Answer 1

You can get the lowest N bits of an address for your personal use by guaranteeing that the objects are aligned to multiples of 1 << N. This can be achieved platform-independently by different ways (alignas and aligned_storage for stack-based objects or std::aligned_alloc for dynamic objects), depending on what you want to achieve:

struct Data { ... };

alignas(1 << 4) Data d; // 4-bits, 16-byte alignment
assert(reinterpret_cast<std::uintptr_t>(&d) % 16 == 0);

// dynamic (preferably with a unique_ptr or alike)
void* ptr = std::aligned_alloc(1 << 4, sizeof(Data));
auto obj = new (ptr) Data;
...
obj->~Data();
std::free(ptr);

You pay by throwing away a lot of memory, exponentionally growing with the number of bits required. Also, if you plan to allocate many of such objects contiguously, such an array won't fit in the processor's cacheline for comparatively small arrays, possibly slowing down the program considerably. This solution therefore is not to scale.

Answer 2

If you're sure that the addresses you are passing around always have certain bits unused, then you could use uintptr_t as a transport type. This is an integer type that maps to pointers in the expected way (and will fail to exist on an obscure platform that offers no such possible map).

There aren't any standard macros but you can roll your own easily enough. The code (sans macros) might look like:

void T_func(uintptr_t t)
{
    uint8_t tag = (t & 7);
    T *ptr = (T *)(t & ~(uintptr_t)7);

    // ...
}

int main()
{
    T *ptr = new T;
    assert( ((uintptr_t)ptr % 8) == 0 );
    T_func( (uintptr_t)ptr + 3 );
}

This may defeat compiler optimizations that involve tracking pointer usage.

Answer 3

Well, GCC at least can compute the size of bit-fields, so you can get portability across platforms (I don't have an MSVC available to test with). You can use this to pack the pointer and tag into an intptr_t, and intptr_t is guaranteed to be able to hold a pointer.

#include <limits.h>
#include <stdio.h>
#include <stdint.h>
#include <stddef.h>
#include <inttypes.h>

struct tagged_ptr
{
  intptr_t ptr : (sizeof(intptr_t)*CHAR_BIT-3);
  intptr_t tag : 3;
};

int main(int argc, char *argv[])
{
  struct tagged_ptr p;

  p.tag = 3;
  p.ptr = (intptr_t)argv[0];

  printf("sizeof(p):              %zu <---WTF MinGW!\n", sizeof p);
  printf("sizeof(p):              %lu\n", (unsigned long int)sizeof p);
  printf("sizeof(void *):         %u\n", (unsigned int)sizeof (void *));
  printf("argv[0]:                %p\n", argv[0]);
  printf("p.tag:                  %" PRIxPTR "\n", p.tag);
  printf("p.ptr:                  %" PRIxPTR "\n", p.ptr);
  printf("(void *)*(intptr_t*)&p: %p\n", (void *)*(intptr_t *)&p);
}

Gives:

$ ./tag.exe
sizeof(p):              zu <---WTF MinGW!
sizeof(p):              8
sizeof(void *):         8
argv[0]:                00000000007613B0
p.tag:                  3
p.ptr:                  7613b0
(void *)*(intptr_t*)&p: 60000000007613B0

I've put the tag at the top, but changing the order of the struct would put it at the bottom. Then shifting the pointer-to-be-stored right by 3 would implement the OP's use case. Probably make macros for access to make it easier.

I also kinda like the struct because you can't accidentally dereference it as if it were a plain pointer.