C++ Copy Raw std::bytes to std::vector

Question

I have an FILE_NOTIFY_INFORMATION struct which i have filled like this:

FILE_NOTIFY_INFORMATION* fni = new FILE_NOTIFY_INFORMATION;
fni->Action = 1;
wcscpy_s(fni->FileName,12, L"example.txt");
fni->FileNameLength = 12;
fni->NextEntryOffset = 0;

I have then castet this Struct to an std::byte*.

auto fni_as_byte = reinterpret_cast<std::byte*>(fni);

Now i want to put this fni_as_byte into an vector of std::vector<std::byte>. Because i need this for testing purpose.

Normally you receive the FILE_NOTIFY_INFORMATION for example from the ReadDirectoryChangesW function. And it's called like this:

std::vector<std::byte> notifyBuffer(1024);
res = ReadDirectoryChangesW(handle, notifyBuffer.data(), static_cast<DWORD>(notifyBuffer.size()), false, FILE_NOTIFY_CHANGE_FILE_NAME, nullptr, &overlapped, nullptr);

So how can i manually copy the castet FILE_NOTIFY_INFORMATION into the std::vector<std::byte>?

`std::vector notifyBuffer{1024};` -- This creates a vector of one element, with that value being 1024. Is this what you want? — PaulMcKenzie, Dec 17 '20 at 06:08
@PaulMcKenzie Have I missed something? Did uniform initalisation change? There's a [constructor accepting a size parameter](https://en.cppreference.com/w/cpp/container/vector/vector) which should be called then – unless I'm not up to date... In any case, this unclearness is the reason I consider uniform initialisation flawed and it should not be used. `notifyBuffer(1024)` or `notifyBuffer({1024})` are so much more obvious... — Aconcagua, Dec 17 '20 at 06:30
@Aconcagua `std::vector` also has a constructor that takes a `std::initializer_list`. That is the constructor that `notifyBuffer{1024}` will call, not the size constructor you are thinking of. If that `initializer_list` constructor didn’t exist, then yes, the size constructor would be called instead. — Remy Lebeau, Dec 17 '20 at 06:43
@RemyLebeau OK, then I have missed a change in the course of the standards. I still recall UI preferring the non-initialiser list constructors, if there are. Another breaking change then. Can you tell which standard changed that? Still even more convinced not to use UI... — Aconcagua, Dec 17 '20 at 06:52
@Aconcagua your recall is incorrect. Since its introduction in C++11, UI has always performed a 2-phase lookup, looking only at `std::initializer_list` constructors first, and if no match is found then looking at all other constructors. — Remy Lebeau, Dec 17 '20 at 07:00
@RemyLebeau I see. Thanks. I recall getting different results when explicitly testing, but must have been a non-conformant compiler then. Still there are ambiguities possible. And own code might break if some library adds a initialiser list constructor that wasn't there before. Not the same, but [this one](https://probablydance.com/2013/02/02/the-problems-with-uniform-initialization/) makes me stay with my mind (*required* double brace syntax for initialiser lists, `v{1024}` with 1024 elements and `v{{1024}}` with one single element would have broken my opposition against, but too late...). — Aconcagua, Dec 17 '20 at 07:35

Remy Lebeau · Accepted Answer · 2020-12-17T08:25:41.437

2

Your code is corrupting memory, as the actual size of FILE_NOTIFY_INFORMATION is variable length. See Why do some structures end with an array of size 1?. You are not allocating enough memory to hold the string you are copying into the FileName. To allocate your FILE_NOTIFY_INFORMATION correctly, you would need to do something more like this instead:

FILE_NOTIFY_INFORMATION* fni = (FILE_NOTIFY_INFORMATION*) new std::byte[offsetof(FILE_NOTIFY_INFORMATION, FileName[12])];
fni->Action = 1;
wcscpy_s(fni->FileName, 12, L"example.txt");
fni->FileNameLength = 11 * sizeof(WCHAR); // <— expressed in BYTEs, not including the null terminator!
fni->NextEntryOffset = 0;
...
delete (std::byte*) fni;

That being said, to copy the FILE_NOTIFY_INFORMATION into a std::vector, you can do it like this:

FILE_NOTIFY_INFORMATION* fni = ...;
// fill fni as needed...
std::vector<std::byte> buffer(offsetof(FILE_NOTIFY_INFORMATION, FileName) + fni->FileNameLength);
std::memcpy(buffer.data(), fni, buffer.size());
// use buffer as needed...

Alternatively, just use the std::vector up front as the memory for the original FILE_NOTIFY_INFORMATION:

std::vector<std::byte> buffer(offsetof(FILE_NOTIFY_INFORMATION, FileName[12]));
auto fni = reinterpret_cast<FILE_NOTIFY_INFORMATION*>(buffer.data());
// fill fni as needed....
// use buffer as needed...

edited Dec 17 '20 at 08:25

answered Dec 17 '20 at 06:13

Remy Lebeau

555,201
31
458
770

Note that the last one will break when `buffer` reaches end of scope, as `fni` doesn't own the data. – JHBonarius Dec 17 '20 at 06:44
@JHBonarius in the 2nd example, `fni` is declared after `buffer`, so `fni` will go out of scope first. The only way that code would break is if that pointer is saved somewhere else, and then used after `buffer` is destroyed. – Remy Lebeau Dec 17 '20 at 06:48
Exactly... what if the OP (or some future reader) decides to put this code in a function and `return fni;`? – JHBonarius Dec 17 '20 at 07:39
@RemyLebeau i tried your first solution but it did not work completly. I was able to put the `fni` inside the vector but if i try to read it back from the `vector` to the `FILE_NOTIFY_INFORMATION` Struct with an `reinterpret_cast` the Filename Parameter inside the `fni` is not filled completly. It seems the `sizeof(*fni)` returns a wrong size of the struct – Kevin Dec 17 '20 at 08:00
@Kevin you are right, I didn’t take into account that the size of `FILE_NOTIFY_INFORMATION` is [variable length](https://devblogs.microsoft.com/oldnewthing/20040826-00/?p=38043). But then, neither does your original code, either. So you are actually corrupting memory. I will update my answers to reflect this. – Remy Lebeau Dec 17 '20 at 08:05
@RemyLebeau would it work if i calculate the size like this: `auto size = sizeof(*fni) + fni->FileNameLength` ? – Kevin Dec 17 '20 at 08:15
@RemyLebeau thanks for the edit and the explanation. I have tried it out but now the `FileName` inside the `FILE_NOTIFY_INFORMATION` is to large if i cast it back from the vector. BEcause the `FileNameLength` is calculatet to 22 but the `FileName` Length is only 12. So 10 extra characters are added to the Filename. – Kevin Dec 17 '20 at 08:35
1

@Kevin Read [the docs](https://learn.microsoft.com/en-us/windows/win32/api/winnt/ns-winnt-file_notify_information). `FileNameLength` is in bytes and does not count the trailing null. And `FileName` is Unicode and does not contain a null character. Now do the math. – j6t Dec 17 '20 at 08:56
1

@Kevin the `FileName` is 11 *characters* (not counting the null terminator) that take up 22 *bytes*. Per the documentation: “*`FileNameLength` -The size of the file name portion of the record, **in bytes. Note that this value does not include the terminating null character**. `FileName` - A variable-length field ... The file name is in the **Unicode character format and is not null-terminated**.*” Windows uses UTF-16 for Unicode strings. If you are having trouble getting the `FILE_NOTIFY_INFORMATION` out of the `vector` then you are likely not taking these facts into account properly. – Remy Lebeau Dec 17 '20 at 09:11
@RemyLebeau @j6t sorry guys im dumb =) in the codebase where i used the `FILE_NOTIFY_INFORMATION` from the `ReadDirectoryChangesW` Function i received the `Filename` by doing the Math correct so sorry for that dumb comment =). `std::wstring wPathName(notIf.FileName, notIf.FileName + (notIf.FileNameLength / sizeof(notIf.FileName)))` – Kevin Dec 17 '20 at 09:13
1

@Kevin `sizeof(notIf.FileName)` should be `sizeof(WCHAR)`. They *happen* to be the same byte size in this situation, but you should write your code to be easier to read and understand. In that same vein, I would also suggest use the `wstring` constructor that takes a size parameter, rather than iterators: `std::wstring wPathName(notIf.FileName, notIf.FileNameLength / sizeof(WCHAR));` – Remy Lebeau Dec 17 '20 at 09:14

C++ Copy Raw std::bytes to std::vector

1 Answers1