0

I have been trying to use the CreateFileMapping and PrefetchVirtualMemory functions provided by Windows to share memory between processes. However, I am having trouble locking the physical memory so that it can be read by other processes. Can anyone help me with this issue? Here are the details of my question.

I am trying to optimize the performance of my processes by keeping around 20GB of resources in memory at all times. To achieve this, I am planning to launch a separate tool process that will hold these resources in memory. This will allow my other processes to read from memory directly instead of having to load them from the hard drive each time, which will speed up their performance.

This is my test code:

bool CreateMemoryMapping(const char* filename)
{
    long long m_hMappingSize = GetFileSize(filename);
    DWORD dwMaxSizeHigh = static_cast<DWORD>(m_hMappingSize >> 32);
    DWORD dwMaxSizeLow = static_cast<DWORD>(m_hMappingSize & 0xFFFFFFFF);
    m_hMapping = CreateFileMappingA(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, dwMaxSizeHigh, dwMaxSizeLow, filename);
    if (INVALID_HANDLE_VALUE == m_hMapping)
    {
        cout << "initial file mapping Handel failed," << endl;
        CloseHandle(m_hMapping);
        CloseHandle(m_hFile);
        return false;
    }

    if (!(m_pBase = MapViewOfFile(m_hMapping, FILE_MAP_ALL_ACCESS, NULL, NULL, NULL)))
    {
        cout << "Mapping a file into the address space of a process failed," << endl;
        CloseHandle(m_hMapping);
        CloseHandle(m_hFile);
        return false;
    }
    unsigned long long start = GetTickCount64();
    FileMappingWriteData(filename,  m_hMappingSize);
    unsigned long long end = GetTickCount64();
    unsigned long long duration = end - start;
    float MByte = (m_hMappingSize / 1024.0) / 1024.0;
    printf("write to sm %f MB:%d ms\n", MByte, duration);
    return true;
}

void FileMappingWriteData(const char* filename,  UINT64 datasize)
{
    /*DWORD dwBytesRead = 0;
    BOOL bSuccess = ReadFile(m_hFile, m_pBase, datasize, &dwBytesRead, NULL);*/
    void* data = malloc(datasize);
    FILE* f = nullptr;
    fopen_s(&f, filename, "rb");
    if (f)
    {
        read(data, datasize, f);
        write(m_pBase, datasize, data);
    }
    WIN32_MEMORY_RANGE_ENTRY  rangeArray;
    rangeArray.VirtualAddress = m_pBase;
    rangeArray.NumberOfBytes = datasize;
    HANDLE hProcess = GetCurrentProcess();
    BOOL success = PrefetchVirtualMemory(hProcess, 1, &rangeArray, 0);

    if (success) cout << "Resident physical memory success";
    Sleep(5000);
    if (VirtualLock(m_pBase, datasize)) cout << "Check to resident physical memory successful";
    CloseHandle(hProcess);
    free(data);
}

There are two issues here. Firstly, I used the GetTickCount64 timestamp to measure the time it takes to read from shared memory and directly from the SSD hard drive. I found that they take the same amount of time. Secondly, I opened the Task Manager and found that both of them produce the same number of page faults. This indicates that the data is not being held in physical memory at all.

The second issue is that I used VirtualLock to check whether a file of tens of MBs has been successfully locked in memory. Although PrefetchVirtualMemory returns true, VirtualLock still cannot detect that it has been successful, even after sleeping for a long time. I checked and found that it should be possible to lock up to 2GB, but mine cannot. I tried locking a resource of 1KB, and both the sender and receiver showed that the physical memory was locked successfully, but the speed was still as slow as directly reading from the hard disk. Additionally, the number of page faults was the same.

I've tried turning on the Locked Memory page in Group Policy and used memory mapping software like Primo cache, but none of them worked.

These questions have been bothering me for a long time and I look forward to getting answers.

Thank you in advance.

Cavin Macwan
  • 1,183
  • 5
  • 12
xd_m687
  • 1
  • 1
  • Is it possible to prefetch 20GB while PrefetchVirtualMemory behaves like [a strong hint by the system and is subject to usual physical memory constraints where it can completely or partially fail under low-memory conditions](https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-prefetchvirtualmemory#remarks)? It seems there is [paging](https://learn.microsoft.com/en-us/windows/win32/memory/file-mapping) behavior for the file while mapped. – YangXiaoPo-MSFT Apr 25 '23 at 08:06
  • And according to [VirtualLock](https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtuallock#remarks) , The maximum number of pages that a process can lock is equal to the number of pages in its minimum working set minus a small overhead. Applications that need to lock larger numbers of pages must first call the `SetProcessWorkingSetSize` function to increase their minimum and maximum working set sizes. – YangXiaoPo-MSFT Apr 25 '23 at 08:32

0 Answers0