SHARING_VIOLATION with multi-threaded file IO on Windows

Question

I have some code that resembles this minimal reproduction example (the real version generates some code and compiles it):

#include <fstream>
#include <string>
#include <thread>
#include <vector>

void write(unsigned int thread)
{
    std::ofstream stream("test_" + std::to_string(thread) + ".txt");
    stream << "test" << std::endl;
    stream << "thread" << std::endl;
    stream << "bad" << std::endl;
}

void test(unsigned int thread)
{
    write(thread);
#ifdef _WIN32
    const std::string command = "rename test_" + std::to_string(thread) + ".txt test_renamed_" + std::to_string(thread) + ".txt";
#else
    const std::string command = "mv test_" + std::to_string(thread) + ".txt test_renamed_" + std::to_string(thread) + ".txt";
#endif
    system(command.c_str());
}

int main()
{
    std::vector<std::thread> threads;
    for(unsigned int i = 0; i < 5; i++) {
        // Remove renamed file
        std::remove(("test_renamed_" + std::to_string(i) + ".txt").c_str());

        threads.emplace_back(test, i);
    }

    // Join all threads
    for(auto &t : threads) {
        t.join();
    }
    return EXIT_SUCCESS;
}

My understanding is that std::ofstream should behave in a nice RAII manner and close and flush at the end of the write function. On Linux, it appears to do just this. However, on Windows 10 I get sporadic "The process cannot access the file because it is being used by another process" errors. I've dug into it with procmon and it looks like the file isn't getting closed by the parent process (22224) resulting in the SHARING_VIOLATION which presumably causes the error: Although the procmon trace looks like the problem is within my process, I have tried turning off the virus scanner. I have also tried using C-style fopen,fprintf,fclose and also ensuring that the process I'm spawning with system isn't inheriting file handles somehow by clearing HANDLE_FLAG_INHERIT on the underlying file handle...which leaves me somewhat out of ideas! Any thoughts SO?

Answer 1

At least on VS 2017, I can confirm the file is closed from your snippet. (In destructor of ofstream, the code calls fclose on the handle).

I think however, that the issue is not in the C++ code, but the behavior of the OS.

With Windows, the act of removing a file which the OS thinks is open, will be blocked. In Unix the behavior of unlinking a file from a directory, is to allow existing handles to continue acting the orphaned files. So in unix the operation could never be a sharing violation, as the act of unlinking a file is a different operation. linux semantics can be opted into for recent windows 10 builds.

procmon on Windows has a given altitude. That means that any operation which is perfformed by virus scanners may be hidden to procmon, and it would give a false answer.

A process can also duplicate a handle on the open file, and that would also cause this issue, but not show the handle being closed.

Answer 2

We can rewrite the file writing using Win32 API:

void writeRaw(unsigned int thread)
{
    const auto str = "test_" + std::to_string(thread) + ".txt";
    auto hFile = CreateFileA(str.c_str(), GENERIC_WRITE, 
        FILE_SHARE_WRITE, nullptr, CREATE_ALWAYS, 0, nullptr);
    DWORD ret{};
    WriteFile(hFile, str.data(), str.size(), &ret, nullptr);
    CloseHandle(hFile);
}

Running the test still gives a file share violation due to the way windows works. When the last handle is closed, filesystem driver performs IRP_MJ_CLEANUP IOCTL to finish processing anything related to the file. Antivirus software, for instance, would attempt to scan the file (and incidentally holds the lock on it =) ). Additionally documentation MSDN IRP_MJ_CLEANUP states that:

It is important to note that when all handles to a file object have been closed, this does not necessarily mean that the file object is no longer being used. System components, such as the Cache Manager and the Memory Manager, might hold outstanding references to the file object. These components can still read to or write from a file, even after an IRP_MJ_CLEANUP request is received.

Conclusion: It is expected that to receive a file share violation in windows if a process tries to do something with the file shortly after closing the handle as the underlying system components are still processing the file close request.

Answer 3

The most probable cause of the problem is that when you delete a file in windows ,it isn't immediatly deleted (it's just flagged for deletion). It can/will take some milliseconds (up to seconds if you're very unlucky) for it to be actually deleted.

Source : Niall Douglas in “Better mutual exclusion on the filesystem using Boost.AFIO" about 10m:10s https://www.youtube.com/watch?v=9l28ax3Zq0w