Branchless memory manager?

Question

Anyone thought about how to write a memory manager (in C++) that is completely branch free? I've written a pool, a stack, a queue, and a linked list (allocating from the pool), but I am wondering how plausible it is to write a branch free general memory manager.

This is all to help make a really reusable framework for doing solid concurrent, in-order CPU, and cache friendly development.

Edit: by branchless I mean without doing direct or indirect function calls, and without using ifs. I've been thinking that I can probably implement something that first changes the requested size to zero for false calls, but haven't really got much more than that. I feel that it's not impossible, but the other aspect of this exercise is then profiling it on said "unfriendly" processors to see if it's worth trying as hard as this to avoid branching.

Answer 1

While I don't think this is a good idea, one solution would be to have pre-allocated buckets of various log₂ sizes, stupid pseudocode:

class Allocator {

    void* malloc(size_t size) {
        int bucket = log2(size + sizeof(int));
        int* pointer = reinterpret_cast<int*>(m_buckets[bucket].back());
        m_buckets[bucket].pop_back();
        *pointer = bucket; //Store which bucket this was allocated from
        return pointer + 1; //Dont overwrite header
    }

    void free(void* pointer) {
        int* temp = reinterpret_cast<int*>(pointer) - 1;
        m_buckets[*temp].push_back(temp);
    }

    vector< vector<void*> > m_buckets;
};

(You would of course also replace the std::vector with a simple array + counter).

EDIT: In order to make this robust (i.e. handle the situation where the bucket is empty) you would have to add some form of branching.

EDIT2: Here's a small branchless log2 function:

//returns the smallest x such that value <= (1 << x)
int
log2(int value) {
    union Foo {
        int x;
        float y;
    } foo;
    foo.y = value - 1;
    return ((foo.x & (0xFF << 23)) >> 23) - 126; //Extract exponent (base 2) of floating point number
}

This gives the correct result for allocations < 33554432 bytes. If you need larger allocations you'll have to switch to doubles.

Here's a link to how floating point numbers are represented in memory.

Answer 2

The only way I know to create a truly branchless allocator is to reserve all the memory it will potentially use in advance. Otherwise there's always going to be some hidden code somewhere to see if we're exceeding some current capacity whether it's in a hidden push_back in a vector checking if the size exceeds capacity used to implement it or something of that sort.

Here is one such crude example of a fixed alloc which has a completely branchless malloc and free method.

class FixedAlloc
{
public:
    FixedAlloc(int element_size, int num_reserve)
    {
        element_size = max(element_size, sizeof(Chunk));
        mem = new char[num_reserve * element_size];

        char* ptr = mem;
        free_chunk = reinterpret_cast<Chunk*>(ptr);
        free_chunk->next = 0;

        Chunk* last_chunk = free_chunk;
        for (int j=1; j < num_reserve; ++j)
        {
            ptr += element_size;
            Chunk* chunk = reinterpret_cast<Chunk*>(ptr);
            chunk->next = 0;
            last_chunk->next = chunk;
            last_chunk = chunk;
        }
    }

    ~FixedAlloc()
    {
        delete[] mem;
    }

    void* malloc()
    {
        assert(free_chunk && free_chunk->next && "Reserve memory exhausted!");
        Chunk* chunk = free_chunk;
        free_chunk = free_chunk->next;
        return chunk->mem;
    }

    void free(void* mem)
    {
        Chunk* chunk = static_cast<Chunk*>(mem);
        chunk->next = free_chunk;
        free_chunk = chunk;
    }

private:
    union Chunk
    {
        Chunk* next;
        char mem[1];
    };
    char* mem;
    Chunk* free_chunk;
};

Since it's totally branchless, it simply segfaults if you try to allocate more memory than initially reserved. It also has undefined behavior for trying to free a null pointer. I also avoided dealing with alignment for the sake of a simpler example.