Why is a placement new much faster than a direct assignment?

Question

I recently figured out that using a placement new is faster than doing 16 assignments:
Consider the following piece of code (c++11):

class Matrix
{
public:
    double data[16];

    Matrix() : data{ 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1 }
    {
    };

    void Identity1()
    {
        new (this) Matrix();
    };

    void Identity2()
    {
        data[0]  = 1.0; data[1]  = 0.0; data[2]  = 0.0; data[3]  = 0.0;
        data[4]  = 0.0; data[5]  = 1.0; data[6]  = 0.0; data[7]  = 0.0;
        data[8]  = 0.0; data[9]  = 0.0; data[10] = 1.0; data[11] = 0.0;
        data[12] = 0.0; data[13] = 0.0; data[14] = 0.0; data[15] = 1.0;
    };
};

Usage:

Matrix m;
//modify m.data

m.Identity1(); //~25 times faster
m.Identity2();

On my machine Identity1() is about 25 times faster than the second function. And now im curious why there is such a big difference?

I also tried a third one:

void Identity3()
{
    memset(data, 0, sizeof(double) * 16);
    data[0] = 1.0;
    data[5] = 1.0;
    data[10] = 1.0;
    data[15] = 1.0;
};

But this is even slower than Identity2() and i can't imagine why.

---

Profiling information

I have done several profiling tests to see if it's an profiling-related issue, so there is the default 'for loop' test but also external profiling tests:

Profiling method 1: (the well known for loop test)

struct timespec ts1;
struct timespec ts2;

clock_gettime(CLOCK_MONOTONIC, &ts1);

for (volatile int i = 0; i < 10000000; i++)
    m.Identity(); //use 1 or 2 here

clock_gettime(CLOCK_MONOTONIC, &ts2);

int64_t start = (int64_t)ts1.tv_sec * 1000000000 + (int64_t)ts1.tv_nsec;
int64_t elapsed = ((int64_t)ts2.tv_sec * 1000000000 + (int64_t)ts2.tv_nsec) - start;

if (elapsed < 0)
    elapsed += (int64_t)0x100000 * 1000000000;

printf("elapsed nanos: %ld\n", elapsed);

Method 2:

$ valgrind --tool=callgrind ./testcase

$ # for better overview:
$ python2 gprof2dot.py -f callgrind.out.22028 -e 0.0 -n 0.0 | dot -Tpng -o tree.png

---

Assembly information

As the user T.C. stated in the comments, this might be helpful:

http://goo.gl/LC0RdG

---

Compilation and machine info

Compiled with: g++ --std=c++11 -O3 -g -pg -Wall

-pg is not the issue. Got the same time-difference in measurement method 1 without using this flag.

Machine info (lscpu):

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 58
Model name:            Intel(R) Core(TM) i7-3612QM CPU @ 2.10GHz
Stepping:              9
CPU MHz:               2889.878
CPU max MHz:           3100.0000
CPU min MHz:           1200.0000
BogoMIPS:              4192.97
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              6144K
NUMA node0 CPU(s):     0-7

Answer 1

Whatever your 25x time difference was measuring, it's not actually the difference between the two Identity() implementations.

With your timing code, both versions compile to exactly the same asm: an empty loop. The code you posted never uses m, so it gets optimized away. All that happens is loads/stores of the loop counter. (This happens because you used volatile int, to tell gcc that the variable is stored in memory-mapped I/O space, so all reads/writes of it appearing in the source must actually appear in the asm. MSVC has a different meaning for the volatile keyword, which goes beyond what the standard says.)

Have a look at the asm on godbolt. Here is your code, and the asm it turns into:

for (volatile int i = 0; i < 10000000; i++)
    m.Identity1();
// same output for gcc 4.8.2 through gcc 5.2.0, with -O3

# some setup before this loop:  mov $0, 8(%rsp)  then test if it reads back as 0
.L16:
    movl    8(%rsp), %eax
    addl    $1, %eax
    movl    %eax, 8(%rsp)
    movl    8(%rsp), %eax
    cmpl    $9999999, %eax
    jle .L16

---

  for (volatile int i = 0; i < 10000000; i++)
    m.Identity2();

# some setup before this loop:  mov $0, 12(%rsp)  then test if it reads back as 0
.L15:
    movl    12(%rsp), %eax
    addl    $1, %eax
    movl    %eax, 12(%rsp)
    movl    12(%rsp), %eax
    cmpl    $9999999, %eax
    jle .L15

As you can see, neither one calls either version of the Identity() function.

It's interesting to see in the asm for Identity1 that it uses integer movq for assignment of zeros, while Identity2 only uses scalar FP moves. This may have something to do with using 0.0 vs. 0, or it may be due to in-place new vs. simple assignment.

I see either way, gcc 5.2.0 doesn't vectorize the Identity functions unless you use -march=native. (In which case it uses AVX 32B loads/stores to copy from 4x 32B of data. Nothing clever like byte-shifting the registers to move the 1.0 to a different spot :/)

If gcc was smarter, it would do a 16B store of two zeros, and do that instead of two movsd. Maybe it's assuming unaligned, and the downside for a cacheline or page-line split on an unaligned store is a lot worse than the upside of saving a store insn if it is aligned.

---

So whatever you timed with that code, it wasn't your functions. Unless one of them did the Identity, and the other didn't. Either way, lose the volatile from your loop counter, that's totally silly. Just look at the extra loads/stores in the empty loops because of it.

Answer 2

I bet you get the same performance if you memcopy a const-expr array manually:

static constexpr double identity_data[16] = { 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1 };

void Identity3()
{
    std::copy(std::begin(identity_data), std::end(identity_data), data);
}

Answer 3

Intrigued by the question, I found a very nice blog post on SSE instructions, discussing the performance of movq and movsd here:

http://www.gamedev.net/blog/615/entry-2250281-demystifying-sse-move-instructions/

Since the second set of instructions [movsd/movsq] don't do zero extension, you might think that they would be slightly faster than ones that have to do the extra filling of zeroes [movd/movq]. However, these instructions can introduce a false dependence on previous instructions, since the processor doesn't know whether you intended to use the extra data you didn't end up erasing. During out-of-order execution, this can cause stalls in the pipeline while the move instruction waits for any previous instructions that have to write to that register. If you didn't actually need this dependence, you've unnecessarily introduced a slowdown into your application.

So the more complex instruction decoding plays ball with the pipeline, where the other instructions have to assume dependencies. The decoding itself is probably just as fast.

Trying out a few things on the assembly page, I was also amazed by how bad a simple memset translates into inline assembly, when all I expected was a simple rep stosq or an unrolled version of that.