How to append in file in Windows in UnBuffered mode using CreateFile

Question

Every time my function is getting called it is overwriting to the file. Kindly note I am opening file in unbuffered mode using below flags.

FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH

If I am using simple buffered mode it is working fine.

FILE_ATTRIBUTE_NORMAL

I am getting following error in unbuffered mode.

** ERROR ** CreateFile failed: The parameter is incorrect.

Kindly find the code snippets below. This piece of code getting called many times.

HANDLE hFile;

LPCWSTR file_path = convertCharArrayToLPCWSTR(UNBUFFERED_FILE);

hFile = CreateFile(file_path,
    FILE_APPEND_DATA,
    FILE_SHARE_WRITE,
    NULL,
    OPEN_ALWAYS,
    FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH,
    NULL
);

if (hFile == INVALID_HANDLE_VALUE)
{
    std::cout << "Unable to open/create file for writing" << std::endl;
    PrintError(TEXT("CreateFile failed"));
}


Data *data = new Data();
DWORD dwBytesToWrite = sizeof(Data);
DWORD dwBytesWritten = 0;
BOOL bErrorFlag = FALSE;


bErrorFlag = WriteFile(
    hFile,                  // open file handle
    data,                   // start of data to write
    dwBytesToWrite,         // number of bytes to write
    &dwBytesWritten,        // number of bytes that were written
    NULL);

if (bErrorFlag == FALSE)
{
    std::cout << "Unable to write to file" << std::endl;
    PrintError(TEXT("Unable to write to file"));
}

if (dwBytesToWrite != dwBytesWritten)
{
    std::cout << "Error in writing: Whole data not written" << std::endl;
    PrintError(TEXT("Error in writing: Whole data not written"));
}

CloseHandle(hFile);

. Kindly suggest if any alternative idea is available.

The code in the example differs from text. So it's hard to tell what set of parameters doesn't work. Please think about minimum working example (MWE). Additionally it might be important to know what kind of file system and media you want to write, probably a WRITE_TRHOUGH is impossible to some media like a network share. One could try it, if she has a MWE. — harper, Sep 25 '18 at 07:29
From the [CreateFile documentation](https://learn.microsoft.com/fr-ch/windows/desktop/api/fileapi/nf-fileapi-createfilea): _There are strict requirements for successfully working with files opened with CreateFile using the FILE_FLAG_NO_BUFFERING flag, for details see [File Buffering](https://learn.microsoft.com/fr-ch/windows/desktop/FileIO/file-buffering)._ Is there any reason for using FILE_FLAG_NO_BUFFERING ? — Jabberwocky, Sep 25 '18 at 07:30
@Jabberwocky - yes. exactly. only on page *File Buffering* exist error *File access buffer addresses for read and write operations should be physical sector-aligned* - this is wrong. really correct info [here](https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/nf-ntifs-ntcreatefile) - *Buffers must be aligned in accordance with the alignment requirement of the underlying device.* this is not sector size but usual 1-2-4 bytes — RbMm, Sep 25 '18 at 07:42
Your code and the error message you get do not agree. Don't post fake code or fake error messages. — IInspectable, Sep 25 '18 at 07:58
@IInspectable the question as it stands now makes sense to me, the error message and the code agree IMO. — Jabberwocky, Sep 25 '18 at 07:59
@jab: They don't. To get an error code that can be translated to *"The parameter is incorrect."* mandates, that the code is calling `GetLastError` at some point. The **vast** majority of calls to `GetLastError` are wrong, and since we cannot see that call, we don't know, whether error code 87 is in fact meaningful. — IInspectable, Sep 25 '18 at 08:13
`FILE_APPEND_DATA` and `FILE_FLAG_NO_BUFFERING` are mutually exclusive — RbMm, Sep 25 '18 at 08:22

score 5 · Accepted Answer · answered Sep 25 '18 at 08:30

5

from NtCreateFile

FILE_NO_INTERMEDIATE_BUFFERING
The file cannot be cached or buffered in a driver's internal buffers. This flag is incompatible with the DesiredAccess parameter's FILE_APPEND_DATA flag.

so when you call

CreateFile(file_path,
    FILE_APPEND_DATA, // !!
    FILE_SHARE_WRITE,
    NULL,
    OPEN_ALWAYS,
    FILE_FLAG_NO_BUFFERING /*!!*/| FILE_FLAG_WRITE_THROUGH,
    NULL
);

you use FILE_FLAG_NO_BUFFERING (mapped to FILE_NO_INTERMEDIATE_BUFFERING) with FILE_APPEND_DATA - you and must got ERROR_INVALID_PARAMETER. you need remove one flag. i suggest remove FILE_FLAG_NO_BUFFERING flag, because with it you can write only integral of the sector size.

answered Sep 25 '18 at 08:30

RbMm

31,280
3
35
56

I tried with one one flag FILE_FLAG_WRITE_THROUGH and it worked for me now Thanks a lot. – Ravindra Gupta Sep 25 '18 at 08:46
Actually initially i was using both the flag. Now I am using only flag FILE_FLAG_WRITE_THROUGH. Which is not giving any error and writing properly. Point of discussion could be that weather it is writing in unbuffered mode or not...as discription it says " Write operations will not go through any intermediate cache, they will go directly to disk." I believe this one doing unbuffered write. What you say? – Ravindra Gupta Sep 25 '18 at 08:56
@RavindraGupta - with `FILE_FLAG_WRITE_THROUGH` data will be written just to disk but to cache also – RbMm Sep 25 '18 at 09:12

How to append in file in Windows in UnBuffered mode using CreateFile

1 Answers1