difference between return address of operator new[] and the actual address got for the array

Question

I'm playing new operator overloading recently. I noticed a strange behavior when I overload new[] operator (the new operator for allocating arrays).

Here is my code:

#include <iostream>
using namespace std;

class Pool
{
public:
    void* alloc(size_t size) {
        return malloc(size);
    }
};

class MyClass
{
public:
    MyClass() {
        cout<<"ctor called"<<endl;
    }
    ~MyClass() {
        cout<<"dtor called"<<endl;
    }
    void* operator new(size_t size) {
        cout<<"new called, size: "<<size<<endl;
        return (void*)malloc(size);
    }
    void* operator new[](size_t size) {
        cout<<"new[] called, size: "<<size<<endl;
        void* result = (void*)malloc(size);
        cout<<"in new[]: "<<result<<endl;
        return result;
    }
    void* operator new(size_t size, void* ptr) {
        cout<<"new(ptr) called, size: "<<size<<endl;
        return (void*)ptr;
    }
    void* operator new(size_t size, Pool& pool) {
        cout<<"new(Pool) called, size: "<<size<<endl;
        return (void*)pool.alloc(size);
    }
    void operator delete(void* ptr) {
        cout<<"delete called, ptr: "<<ptr<<endl;
        free(ptr);
    }
    void operator delete(void* ptr, size_t size) {
        cout<<"delete called, ptr: "<<ptr<<", size: "<<size<<endl;
        free(ptr);
    }
    void operator delete[](void* ptr) {
        cout<<"delete[] called, ptr: "<<ptr<<endl;
        free(ptr);
    }
    void operator delete[](void* ptr, size_t size) {
        cout<<"delete[] called, ptr: "<<ptr<<", size: "<<size<<endl;
        free(ptr);
    }
    uint32_t data;
};

int main() {
    Pool pool;
    cout<<"Pool"<<endl;
    new Pool;
    cout<<"MyClass"<<endl;
    MyClass *ptr1, *ptr2, *ptr3;
    ptr1 = new MyClass;
    ptr2 = new MyClass[10]();
    cout<<(void*)ptr2<<endl;
    ptr3 = new(pool) MyClass;
    delete ptr1;
    delete[] ptr2;
    delete ptr3;

    return 0;
}

And the result (with gcc 64bit on OS X) is like:

Pool
MyClass
new called, size: 4
ctor called
new[] called, size: 48
in new[]: 0x7fa7f0403840
ctor called
ctor called
ctor called
ctor called
ctor called
ctor called
ctor called
ctor called
ctor called
ctor called
0x7fa7f0403848
new(Pool) called, size: 4
ctor called
dtor called
delete called, ptr: 0x7fa7f0403830
dtor called
dtor called
dtor called
dtor called
dtor called
dtor called
dtor called
dtor called
dtor called
dtor called
delete[] called, ptr: 0x7fa7f0403840
dtor called
delete called, ptr: 0x7fa7f0403870

I noticed three things: 1st, I asked to allocate 10 objects of 4 bytes in new[], but the actual request received by the function is 48 bytes. 2nd, apparently the first 8 bytes are used for other purpose: the actual address received by ptr2 is 8 bytes after the address returned by the new[] operator. 3rd, the address is also automatically translated (by going forward 8 bytes) in the overloaded delete[] function.

I also noticed that this behavior happens only when I explicitly implement the destructor. If I only use the default destructor, the 8 bytes are just gone.

Can anyone tell me what is happening behind this? What are the 8 bytes used for?

Thanks.

score 3 · Accepted Answer · edited May 23 '17 at 12:10

The array-new expression is allowed to call the array-operator-new with more space than needed for the array. All that's required is that the value of the array-new expression is a pointer to the first element in the array.

Practically, the extra space is needed to store information on how many elements need to be destroyed when the array is destroyed (sometimes called an "array cookie").

It's interesting to note that the actual amount of extra memory that is requested from the array-operator-new function is completely unknowable and may change with every call. This basically makes the array-placement-new expression defective and unusable.

Just for reference, the relevant clause is C++11 5.3.4/10:

A new-expression passes the amount of space requested to the allocation function as the first argument of type std::size_t. That argument shall be no less than the size of the object being created; it may be greater than the size of the object being created only if the object is an array.

The most interesting example follows just below:

new T[5] results in a call of operator new[](sizeof(T) * 5 + x), and

new(2,f) T[5] results in a call of operator new[](sizeof(T) * 5 + y, 2, f).

Here, x and y are non-negative unspecified values representing array allocation overhead; the result of the new-expression will be offset by this amount from the value returned by operator new[]. This overhead may be applied in all array new-expressions, including those referencing the library function operator new[](std::size_t, void*) and other placement allocation functions. The amount of overhead may vary from one invocation of new to another.

You may be pleased to learn that the Itanium ABI has very sensible rules about array cookies; for example, none are needed for arrays of trivially-destructible objects.

Thanks a lot. Still I'm confused why just adding a destructor would require such overhead. — tomzhi, Oct 13 '13 at 22:42
Yes, I understand. But if I remove the destructor declaration (using only the default one generated by the compiler), there would be only 40 bytes required. The extra 8 bytes are gone. How would someone know how many objects are allocated in this case? — tomzhi, Oct 13 '13 at 22:53
@JimmyJi: if the destructor does nothing, then it doesn't need to get called, so there's no need to keep the element count around. — Kerrek SB, Oct 13 '13 at 22:57

difference between return address of operator new[] and the actual address got for the array

1 Answers1