1

I'm reading the NTFS change Journal to monitor any file changes on a drive but the USN_RECORD's returned only include the filename of the associated file and not the full path.

To get the full path I'm using the FileReferenceNumber (Also in the USN_RECORD) with OpenFileById API to get a handle to the associated file, followed by GetFinalPathNameByHandleA to get the full path.

This works great as long that file still exists, but if the file has been deleted (For example if I'm processing a USN_RECORD for a file deletion) then obviously OpenFileById fails and so I'm unable to get the full path.

Whilst the USN_RECORD does include a ParentFileReferenceNumber member (Which I could presumably open using OpenFileById) that should allow me to get the name of the parent directory, the same problem then exists, i.e. what if the parent directory had been deleted?

Also even if the parent directory hasn't been deleted how do I then walk up to the next directory up (i.e. the parent of the parent) and so on until I reach to volume root?

Here is my example code:

#include "stdafx.h"
#include <Windows.h>
#include <WinIoCtl.h>
#include <stdio.h>
#include <string>

using std::string;
#define BUF_LEN 4096

std::string GetFullPath(HANDLE hVol, DWORDLONG fileRefNum)
{
    FILE_ID_DESCRIPTOR fid;
    ZeroMemory(&fid, sizeof(fid));
    fid.dwSize = sizeof(fid);
    fid.Type = FileIdType;
    fid.FileId.QuadPart = fileRefNum;   

    HANDLE handle = OpenFileById(hVol, &fid, 0, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, FILE_FLAG_BACKUP_SEMANTICS);
    if (handle == INVALID_HANDLE_VALUE)
        return "Error: " + std::to_string(GetLastError());

    char buffer[1024];
    GetFinalPathNameByHandleA(handle, &buffer[0], 1024, 0);

    return string(&buffer[0]);
}

void main()
{
    // Open the volume
    DWORD dwBytes;
    DWORD dwRetBytes;
    HANDLE hVol = CreateFile(TEXT("\\\\.\\c:"), GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL);
    if (hVol == INVALID_HANDLE_VALUE)
    {
        printf("CreateFile failed (%d)\n", GetLastError());
        return;
    }

    // Query the Journal
    USN_JOURNAL_DATA_V0 JournalData;
    if (!DeviceIoControl(hVol, FSCTL_QUERY_USN_JOURNAL, NULL, 0, &JournalData, sizeof(JournalData), &dwBytes, NULL))
    {
        printf("Query journal failed (%d)\n", GetLastError());
        return;
    }


    printf("Journal ID: %I64x\n", JournalData.UsnJournalID);
    printf("FirstUsn: %I64x\n\n", JournalData.FirstUsn);

    READ_USN_JOURNAL_DATA_V0 ReadData;
    ZeroMemory(&ReadData, sizeof(ReadData));
    ReadData.ReasonMask = 0xFFFFFFFF;
    ReadData.UsnJournalID = JournalData.UsnJournalID;
    PUSN_RECORD UsnRecord;
    CHAR Buffer[BUF_LEN];
    for (int loop = 0; loop <= 100; loop++)
    {
        // Read the journal
        ZeroMemory(Buffer, BUF_LEN);
        if (!DeviceIoControl(hVol, FSCTL_READ_USN_JOURNAL, &ReadData, sizeof(ReadData), &Buffer, BUF_LEN, &dwBytes, NULL))
        {
            printf("Read journal failed (%d)\n", GetLastError());
            return;
        }
        dwRetBytes = dwBytes - sizeof(USN);

        // Find the first record
        UsnRecord = (PUSN_RECORD)(((PUCHAR)Buffer) + sizeof(USN));
        printf("****************************************\n");

        // This loop could go on for a long time, given the current buffer size.
        while (dwRetBytes > 0)
        {
            printf("USN:            %I64x\n", UsnRecord->Usn);
            printf("Filename:       %.*S\n", UsnRecord->FileNameLength / 2, UsnRecord->FileName);
            printf("Full filename:  %s\n", GetFullPath(hVol, UsnRecord->FileReferenceNumber).c_str());
            printf("Reason:         %x\n", UsnRecord->Reason);
            printf("\n");

            dwRetBytes -= UsnRecord->RecordLength;

            // Find the next record
            UsnRecord = (PUSN_RECORD)(((PCHAR)UsnRecord) + UsnRecord->RecordLength);
        }
        // Update starting USN for next call
        ReadData.StartUsn = *(USN *)&Buffer;
    }

    CloseHandle(hVol);
}

And example output:

USN:            8ebc423a8
Filename:       999000000167237.xml
Full filename:  Error: 87
Reason:         80000200

USN:            8ebc42410
Filename:       999000000167238.xml
Full filename:  Error: 87
Reason:         80000200

USN:            8ebc42478
Filename:       MobilityDB.db
Full filename:  \\?\C:\ProgramData\NGC\Open Mobile\Profiles\12316\MobilityDB.db
Reason:         1
BenS1
  • 183
  • 2
  • 11
  • really even for not deleted files you can got already not valid `FileReferenceNumber` in usn record. open file by it and return `ERROR_INVALID_PARAMETER` (87). and as general note `GetFinalPathNameByHandleA` very not effective way for get full path. need use `GetFileInformationByHandleEx` with `FileNameInfo`. however not every file you can open. i think only option here make full files list by `FSCTL_ENUM_USN_DATA` than sort it and yourself search parent of every file. say in my test i found parents for 166774 files and not found for 47 on some disk – RbMm Oct 31 '17 at 20:04
  • Thanks for your reply @RbMm. What makes you think that the FileReferenceNumber may not be valid for non-deleted files? I've not seen this mentioned anywhere before. Similarly why isn't GetFinalPathNameByHandle an effective way to get the full path, and why is using GetFileInformationByHandleEx considered better? Are there known issues with GetFinalPathNameByHandle? – BenS1 Nov 01 '17 at 07:53
  • 1
    1 - simply by test, when I enumerate usn records - I view records without `USN_REASON_FILE_DELETE` but it id already not valid or belong to another file. however may be *another* usn record exist for this file already with `USN_REASON_FILE_DELETE`. about `GetFileInformationByHandleEx` - it do single call to file system and return full path inside file system. the `GetFinalPathNameByHandle` do many calls - internally it called `GetFileInformationByHandleEx` too, also called `ZwQueryObject` (and this again call to fs), open,call and close mount manager.. so simply permofance - huge different – RbMm Nov 01 '17 at 08:10
  • Thanks again @RbMm. Yes that was my thoughts too that even though you may process a record a reason other than USN_REASON_FILE_DELETE, it could still be that there is another USN_REASON_FILE_DELETE record later in the Journal. – BenS1 Nov 01 '17 at 09:33
  • Interestingly I just tried using both GetFinalPathNameByHandle and GetFileInformationByHandleEx and I found one file I can successfully open (Using OpenFileById) yet fails when with an error 2 (File not found) when I call GetFinalPathNameByHandle, yet succeeds when I call (Using the same handle) GetFileInformationByHandleEx! I can see the file in Explorer, so I'm not sure why the GetFinalPathNameByHandle is failing. Strange. – BenS1 Nov 01 '17 at 09:33
  • @RbMm, "id already not valid or belong to another file". I don't think FileReferenceNumbers are reused as they include an incrementing sequence number [link](https://jmharkness.wordpress.com/2011/01/27/mft-file-reference-number/) – BenS1 Nov 01 '17 at 09:45
  • yes, full 64bit no, *MFT Record number* only (i use `FSCTL_GET_NTFS_FILE_RECORD` for this). however try open file by id here bad idea anyway. we can use `FSCTL_ENUM_USN_DATA` - `MFT_ENUM_DATA_V1` let set direct `StartFileReferenceNumber` for query. so we can do this by parent id chain. however i be got full list of usn records first and query path yourself – RbMm Nov 01 '17 at 10:10
  • here question is next - for what you do all this ? are you query all records ? if yes - not hard build own list and query full path direct yourself – RbMm Nov 01 '17 at 10:11

0 Answers0