Why is std::fstream faster than WriteFile on Windows

Question

I have tried all sorts of different combinations of flags such as FILE_FLAG_NO_BUFFERING and FILE_FLAG_OVERLAPPED, but fstream::write still beats the Windows API version.

Does std::fstream use internal buffering or other tricks or am I just messing up something?

#include <Windows.h>
#include <chrono>
#include <fstream>
#include <iostream>

std::string createTempFileName()
{
    char buf[800];
    tmpnam_s(buf, sizeof(buf));
    return buf;
}

using namespace std::chrono;

int main()
{
    std::uint64_t count = 1 << 23;

    std::cout << "test fstream\n";
    {
        auto start = steady_clock::now();
        auto path = createTempFileName();
        std::fstream fs(path, std::ios_base::out | std::ios_base::in | std::ios_base::trunc | std::ios_base::binary);
        for (std::uint64_t i = 0; i < count; i++)
            fs.write((char*)&i, sizeof(i));
        fs.close();
        DeleteFile(path.c_str());
        auto end = steady_clock::now();
        std::cout << "fstream: Elapsed time in milliseconds : " << duration_cast<milliseconds>(end - start).count() << " ms\n";
    }

    std::cout << "test WriteFile\n";
    {
        auto start = steady_clock::now();
        auto path = createTempFileName();
        HANDLE file = CreateFile(path.c_str(), GENERIC_WRITE, 0, 0, CREATE_ALWAYS, FILE_FLAG_NO_BUFFERING, NULL);
        for (std::uint64_t i = 0; i < count; i++)
            WriteFile(file, &i, sizeof(i), NULL, NULL);
        CloseHandle(file);
        DeleteFile(path.c_str());
        auto end = steady_clock::now();
        std::cout << "WriteFile: Elapsed time in milliseconds : " << duration_cast<milliseconds>(end - start).count() << " ms\n";
    }
}

Answer 1

I suppose the obvious answer is to look at the source shipped with Visual Studio but since you chose not to do that, let us use our magic thinking hats instead.

There are not that many documented ways to write to a file on Windows and the C/C++ run-time will be using some Windows API to write. Ranked by popularity I would guess we are looking at WriteFile, memory mapped files and IStream. IStream is COM and too high level. Memory mapped files are annoying when you don't know the final size so WriteFile is the most likely candidate.

WriteFile does some minor work in user mode but at the end of the day it is going to end up with a context switch to kernel mode.

In your loop you are only writing 8 bytes every time and therefore a really large part of the CPU time is going to be spent switching in and out of kernel mode and whatever buffering the kernel (and storage hardware) is doing for this file handle does not prevent that.

You are passing the FILE_FLAG_NO_BUFFERING flag to CreateFile yet fail to do any of the required work to make these writes aligned and you fail to check the return value of WriteFile! It might be failing and this entire test could be invalid for all we know.

The C/C++ run-time is often willing to choose speed over size/memory usage and the magic word here is buffering. Even a tiny 16 byte buffer would probably almost double your speed in this specific instance. You can try to turn off this buffering with something like this:

std::fstream fs;
fs.rdbuf()->pubsetbuf(NULL, 0);
fs.open(...