How to get a reliable memory usage information for a 64-bit process from a 32-bit process?

Question

My goal is to get memory usage information for an arbitrary process. I do the following from my 32-bit process:

HANDLE hProc = ::OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_VM_READ, 0, pid);
if(hProc)
{
    PROCESS_MEMORY_COUNTERS_EX pmx = {0};
    if(::GetProcessMemoryInfo(hProc, (PROCESS_MEMORY_COUNTERS*)&pmx, sizeof(pmx)))
    {
        wprintf(L"Working set: %.02f MB\n", pmx.WorkingSetSize / (1024.0 * 1024.0));
        wprintf(L"Private bytes: %.02f MB\n", pmx.PrivateUsage / (1024.0 * 1024.0));
    }

    ::CloseHandle(hProc);
}

The issue is that if the pid process is a 64-bit process, it may have allocated more than 4GB of memory, which will overflow both pmx.WorkingSetSize and pmx.PrivateUsage, which are both 32-bit variables in a 32-bit process. So in that case instead of failing, GetProcessMemoryInfo suceeds with both metrics returned as UINT_MAX -- which is wrong!

So I was wondering, if there was a reliable API to retrieve memory usage from an arbitrary process in a 32-bit application?

No API AFAIK. The best thing you can do is execute a 64-bit process from your 32-bit process to get the info and capture the output. — mnistic, Dec 01 '17 at 01:27
usual processes not deal with another *arbitrary* processes in system. this do only special utilites, design for show system info, debuggers, etc. this kind of utilites almost always must be native bit-ness. so must be 2 separate build - for 32 and 64 bit windows — RbMm, Dec 01 '17 at 01:46

score 2 · Answer 1 · answered Nov 08 '18 at 16:53

There's a reliable API called the "Performance Data Helpers".

Windows' stock perfmon utility is the classical example of a Windows Performance Counter application. Also Process Explorer is using it for collecting process statistics.

Its advantage is that you don't even need the SeDebugPrivilege to gain PROCESS_VM_READ access to other processes.
Note though that access is limited to users being part of the Performance Monitoring Users group.

The idea behind PDH is:

A Query object
- One or multiple counters
Create samples upon request or periodically
Fetch the data you have asked for

It's a bit more work to get you started, but still easy in the end. What I do is setting up a permanent PDH query, such that I can reuse it throughout the lifetime of my application.

There's one drawback: By default, the Operating System creates numbered entries for processes having the same name. Those numbered entries even change while processes terminate or new ones get created. So you have to account for this and cross-check the process ID (PID), actually while having a handle open to the process(es) you want to obtain memory usage for.

Below you find a simple PDH wrapper alternative to GetProcessMemoryInfo(). Of course there's plenty of room to tweak the following code or adjust it to your needs. I have also seen people who created more generic C++ wrappers already.

Declaration

#include <tuple>
#include <array>
#include <vector>
#include <stdint.h>
#include <Pdh.h>

#pragma comment(lib, "Pdh.lib")


class process_memory_info
{
private:
    using pd_t = std::tuple<DWORD, ULONGLONG, ULONGLONG>; // performance data type
    static constexpr size_t pidIdx = 0;
    static constexpr size_t wsIdx = 1;
    static constexpr size_t pbIdx = 2;
    struct less_pd
    {
        bool operator ()(const pd_t& left, const pd_t& right) const
        {
            return std::get<pidIdx>(left) < std::get<pidIdx>(right);
        }
    };

public:
    ~process_memory_info();

    bool setup_query();
    bool take_sample();
    std::pair<uintmax_t, uintmax_t> get_memory_info(DWORD pid) const;

private:
    PDH_HQUERY pdhQuery_ = nullptr;
    std::array<PDH_HCOUNTER, std::tuple_size_v<pd_t>> pdhCounters_ = {};
    std::vector<pd_t> perfData_;
};

Implementation

#include <memory>
#include <execution>
#include <algorithm>
#include <stdlib.h>

using std::unique_ptr;
using std::pair;
using std::array;
using std::make_unique;
using std::get;


process_memory_info::~process_memory_info()
{
    PdhCloseQuery(pdhQuery_);
}

bool process_memory_info::setup_query()
{
    if (pdhQuery_)
        return true;
    if (PdhOpenQuery(nullptr, 0, &pdhQuery_))
        return false;

    size_t i = 0;
    for (auto& counterPath : array<PDH_COUNTER_PATH_ELEMENTS, std::tuple_size_v<pd_t>>{ {
        { nullptr, L"Process", L"*", nullptr, 0, L"ID Process" },
        { nullptr, L"Process", L"*", nullptr, 0, L"Working Set" },
        { nullptr, L"Process", L"*", nullptr, 0, L"Private Bytes" }
        }})
    {
        wchar_t pathStr[PDH_MAX_COUNTER_PATH] = {};

        DWORD size;
        PdhMakeCounterPath(&counterPath, pathStr, &(size = _countof(pathStr)), 0);
        PdhAddEnglishCounter(pdhQuery_, pathStr, 0, &pdhCounters_[i++]);
    }

    return true;
}

bool process_memory_info::take_sample()
{
    if (PdhCollectQueryData(pdhQuery_))
        return false;

    DWORD nItems = 0;
    DWORD size;
    PdhGetFormattedCounterArray(pdhCounters_[0], PDH_FMT_LONG, &(size = 0), &nItems, nullptr);
    auto valuesBuf = make_unique<BYTE[]>(size);
    PdhGetFormattedCounterArray(pdhCounters_[0], PDH_FMT_LONG, &size, &nItems, PPDH_FMT_COUNTERVALUE_ITEM(valuesBuf.get()));
    unique_ptr<PDH_FMT_COUNTERVALUE_ITEM[]> pidValues{ PPDH_FMT_COUNTERVALUE_ITEM(valuesBuf.release()) };

    valuesBuf = make_unique<BYTE[]>(size);
    PdhGetFormattedCounterArray(pdhCounters_[1], PDH_FMT_LARGE, &size, &nItems, PPDH_FMT_COUNTERVALUE_ITEM(valuesBuf.get()));
    unique_ptr<PDH_FMT_COUNTERVALUE_ITEM[]> wsValues{ PPDH_FMT_COUNTERVALUE_ITEM(valuesBuf.release()) };

    valuesBuf = make_unique<BYTE[]>(size);
    PdhGetFormattedCounterArray(pdhCounters_[2], PDH_FMT_LARGE, &size, &nItems, PPDH_FMT_COUNTERVALUE_ITEM(valuesBuf.get()));
    unique_ptr<PDH_FMT_COUNTERVALUE_ITEM[]> pbValues{ PPDH_FMT_COUNTERVALUE_ITEM(valuesBuf.release()) };

    perfData_.clear();
    perfData_.reserve(nItems);
    for (size_t i = 0, n = nItems; i < n; ++i)
    {
        perfData_.emplace_back(pidValues[i].FmtValue.longValue, wsValues[i].FmtValue.largeValue, pbValues[i].FmtValue.largeValue);
    }
    std::sort(std::execution::par_unseq, perfData_.begin(), perfData_.end(), less_pd{});

    return true;
}

pair<uintmax_t, uintmax_t> process_memory_info::get_memory_info(DWORD pid) const
{
    auto it = std::lower_bound(perfData_.cbegin(), perfData_.cend(), pd_t{ pid, 0, 0 }, less_pd{});

    if (it != perfData_.cend() && get<pidIdx>(*it) == pid)
        return { get<wsIdx>(*it), get<pbIdx>(*it) };
    else
        return {};
}


int main()
{
    process_memory_info pmi;
    pmi.setup_query();

    DWORD pid = 4;

    pmi.take_sample();
    auto[workingSet, privateBytes] = pmi.get_memory_info(pid);

    return 0;
}

Thanks for sharing. Some questions: 1) How localizable is it? I always cringe when I have to deal with (English) strings to obtain metrics. 2) What's the overhead of running it vs. `GetProcessMemoryInfo()` call? 3) What minimum OS does it run on? and 4) Can you elaborate on the process being a member of the `"Performance Monitoring Users group"`? That last one could be a real deal-killer. Btw, I ended up resolving my original question by creating my own 64-bit process if running on a 64-bit OS. It also resolved additional issues associated with the wrong bitness. — c00000fd, Nov 08 '18 at 17:38
1) What exactly do you want to localise? Those english strings are just to setup the counters, but won't make it to the outside. 2) I don't know the overhead in comparison to GetProcessMemoryInfo(). I assume it's much higher when you only have a few processes to query. 3) It's available since Win XP/Server 2003. 4) I don't think it's a deal-killer in comparison to PROCESS_VM_READ+SeDebugPrivilege. I have taken this information from [MSDN](https://learn.microsoft.com/en-us/windows/desktop/PerfCtrs/limited-user-access-support). According to my tests, regular users can access the data. — klaus triendl, Nov 08 '18 at 20:26
Putting a Windows user into a user group is a much more complicated as well as a **global** procedure than giving your own process the `SeDebugPrivilege` privilege. As for localization, I meant that `"ID Process"` string wouldn't become something like `"Processus d'identification"` in French, would it? — c00000fd, Nov 08 '18 at 20:49
The paths can be localised as Microsoft's idea was to let users choose what they want to look at (see perfmon). In this case I am using `PdhAddEnglishCounter` for setting up the query, and it works here on a german OS. `SeDebugPrivilege` is of course simple to enable. But this is only feasible if it's already in the process token - AFAIK only when running elevated - AND your process is at high integrity level. [I have an elevated application, which relaunches itself at medium integrity due to UIPI - that's why I was in need of PDH in the first place.] — klaus triendl, Nov 09 '18 at 08:40

score 0 · Answer 2 · answered Dec 01 '17 at 19:41

0

Why don't you compile this application as 64-bit and then you should be able to collect the memory usage for both 32-bit and 64-bit processes.

answered Dec 01 '17 at 19:41

Amit Rastogi

926
2
12
22

Besides the obvious difficulties of re-compiling an old & extensive 32-bit application onto x64 platform, I don't want to do this also because I want my application to run on 32-bit & 64-bit OS in a single image file. – c00000fd Dec 02 '17 at 04:27

score 0 · Answer 3 · answered Dec 02 '17 at 07:45

0

The WMI Win32_Process provider has quite a few 64-bit memory numbers. Not sure if everything you are after is there or not.

answered Dec 02 '17 at 07:45

SoronelHaetir

14,104
1
12
23

How to get a reliable memory usage information for a 64-bit process from a 32-bit process?

3 Answers3

Declaration

Implementation