Why does Kernel32 OpenProcess function return null?

Question

I'm trying to make an application that reads the memory of another (non-Java & 32bit) application using JNA. So far I know how to find process ID and base address of modules. And right before reading memory I need to open process and the OpenProcess function simply returns null. Also, I'm using Windows 10.

    // process id (pid) is known

    final int PROCESS_VM_READ=0x0010;
    final int PROCESS_QUERY_INFORMATION=0x0400;
      
    WinNT.HANDLE processHandle = Kernel32.INSTANCE.OpenProcess(PROCESS_VM_READ | PROCESS_QUERY_INFORMATION, true, pid);

How can I get a process handle?

`Kernel32.GetLastError()` or better `ntdll.RtlGetLastNtStatus()` say you why. most common reason - access denied and invalid process id — RbMm, Oct 04 '20 at 14:06
Okay I checked this and I'm getting access denied. How do I get this access then? — Miłosz Tomkiel, Oct 04 '20 at 14:10
if you have debug privilege - enable it. this must help in your case — RbMm, Oct 04 '20 at 14:15

Daniel Widdis · Accepted Answer · 2020-10-24T19:07:00.643

You need to enable debug privilege for your current process in order to query information for processes owned by anyone other than the current user. The link shows the code in C, but you can port that code to JNA.

This is a one-time method call when your program starts up.

Here's how I do it (hat tip to @RbMm for improvements):

/**
 * Enables debug privileges for this process, required for OpenProcess() to get
 * processes other than the current user
 *
 * @return {@code true} if debug privileges were successfully enabled.
 */
private static boolean enableDebugPrivilege() {
    HANDLEByReference hToken = new HANDLEByReference();
    boolean success = Advapi32.INSTANCE.OpenProcessToken(Kernel32.INSTANCE.GetCurrentProcess(),
            WinNT.TOKEN_QUERY | WinNT.TOKEN_ADJUST_PRIVILEGES, hToken);
    if (!success) {
        LOG.error("OpenProcessToken failed. Error: {}", Native.getLastError());
        return false;
    }
    try {
        WinNT.LUID luid = new WinNT.LUID();
        success = Advapi32.INSTANCE.LookupPrivilegeValue(null, WinNT.SE_DEBUG_NAME, luid);
        if (!success) {
            LOG.error("LookupPrivilegeValue failed. Error: {}", Native.getLastError());
            return false;
        }
        WinNT.TOKEN_PRIVILEGES tkp = new WinNT.TOKEN_PRIVILEGES(1);
        tkp.Privileges[0] = new WinNT.LUID_AND_ATTRIBUTES(luid, new DWORD(WinNT.SE_PRIVILEGE_ENABLED));
        success = Advapi32.INSTANCE.AdjustTokenPrivileges(hToken.getValue(), false, tkp, 0, null, null);
        int err = Native.getLastError();
        if (!success) {
            LOG.error("AdjustTokenPrivileges failed. Error: {}", err);
            return false;
        } else if (err == WinError.ERROR_NOT_ALL_ASSIGNED) {
            LOG.debug("Debug privileges not enabled.");
            return false;
        }
    } finally {
        Kernel32.INSTANCE.CloseHandle(hToken.getValue());
    }
    return true;
}

the end of code is wrong. `AdjustTokenPrivileges` return true even if it fail. bad designed api. need call `GetLastError()` always here for check result. however more simply do this by single call `BOOLEAN b; RtlAdjustPrivilege(SE_DEBUG_PRIVILEGE, TRUE, FALSE, &b);` — RbMm, Oct 04 '20 at 16:59
@RbMm Thanks for the pointer. I'll adjust. Howver, regarding "AdjustTokenPrivileges return true even if it fail", that is only true in the case where it partially succeeds. However, when setting a single privilege that's an impossible outcome, so it will either completely succeed or completely fail. I do not see `RtlAdjustPrivilege()` documented in the Windows API. Using undocumented functions is a poor practice. — Daniel Widdis, Oct 04 '20 at 17:13
*However, when setting a single privilege that's an impossible outcome, so it will either completely succeed or completely fail.* - you mistake here. even if you try set single privilege, which not exist in token - return code will be `STATUS_NOT_ALL_ASSIGNED` (0x106) and `AdjustTokenPrivileges` return true even in this case. only `GetLastError()` return final answer to you — RbMm, Oct 04 '20 at 17:50
can test this for example with `SE_TCB_PRIVILEGE` (usually only localsystem have it) - `AdjustTokenPrivileges` anyway return true here. — RbMm, Oct 04 '20 at 17:57
really `if (Native.getLastError() == 0)` can be condition check. the return value of `AdjustTokenPrivileges` can be complete ignored. always need call `Native.getLastError()` and check it result. also you use string `SE_DEBUG_NAME` and convert it to `LUID` (`SE_DEBUG_PRIVILEGE`). despite this value is well known and documented in *wdm.h*. your code is in general ok and standard. i simply make note about check of ok/fail. `RtlAdjustPrivelege` let do all this via single api call, but formal it undocumented — RbMm, Oct 04 '20 at 18:31

Why does Kernel32 OpenProcess function return null?

1 Answers1

Linked