Is there a way to get DisableUserModeCallbackFilter to work in Windows 10?

Question

Is there any way to get DisableUserModeCallbackFilter (or similar) to work on Windows 10?

It's supposed to allow exceptions thrown from user-mode code to propagate across user/kernel boundaries, and it had a hotfix on earlier Windows versions up to Windows 7, but I can't seem seem to be able to get it working on more recent versions.

Here's a test program that seems to error on Windows 10 x64, but not on Windows XP x86:

#include <tchar.h>
#include <stdio.h>
#include <Windows.h>

#pragma comment(lib, "user32")

WNDPROC oldproc = NULL;

LRESULT CALLBACK newproc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
    _ftprintf(stderr, _T("OMG\n"));
    fflush(stderr);
    throw 0;
    return oldproc(hwnd, uMsg, wParam, lParam);
}

int _tmain(int argc, TCHAR *argv[])
{
    HWND hWnd = CreateWindowEx(0, TEXT("STATIC"), TEXT("Name"),
        WS_OVERLAPPEDWINDOW | WS_VISIBLE,
        CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT,
        NULL, NULL, NULL, NULL);
    oldproc = (WNDPROC)SetWindowLongPtr(hWnd, GWLP_WNDPROC, (LONG_PTR)newproc);
    try {
        UpdateWindow(hWnd);
    } catch (int ex) {
        _ftprintf(stderr, _T("Error: %d\n"), ex);
        fflush(stderr);
    }
}

Entering dangerous territory, tread carefully. [When you transfer control across stack frames, all the frames in between need to be in on the joke](https://blogs.msdn.microsoft.com/oldnewthing/20120910-00/?p=6653). — IInspectable, Mar 19 '18 at 22:12
*across user/kernel boundaries* - you are about ? exception passed to user mode as usual (if not handled by debugger before). problem in `KiUserCallbackDispatcherHandler` (this is user mode routine from *ntdll.dll*). it behavior depend from `RtlGetCurrentPeb()->ProcessParameters->Flags` - 0x80000 flag. the `SetProcessUserModeExceptionPolicy` set or reset this flag now it not exported but you can set this flag yourself (`RtlGetCurrentPeb()->ProcessParameters->Flags |= 0x80000;`) after this will be no fatal error, but your handler anyway will be not called — RbMm, Mar 20 '18 at 01:58
It is a hack to make Win7/Server2008 behave better, default behavior is rather dreadful. They found a much better solution in Win8, re-raising 0xC000041D is pretty effective. Just don't try to defeat a good solution. Targeting x64 is a good solution as well. — Hans Passant, Mar 20 '18 at 15:22
@HansPassant: confused, targeting x64 doesn't make any difference here though? Either way the exception handler at the top isn't called. — user541686, Mar 20 '18 at 21:10

score 2 · Answer 1 · edited Jun 20 '20 at 09:12

when window procedure called - kernel push additional stack frame in kernel stack and in user mode called special "function" (faster even label then normal function) KiUserCallbackDispatcher, which call window procedure and finally return to kernel by special api call ZwCallbackReturn pop kernel stack frame. note that after call ZwCallbackReturn we return not to next instruction after it, but in place from where we enter to kernel, from where user mode callback will be called (usually from GetMessage or PeekMessage).

anyway we must pop kernel stack frame - so call ZwCallbackReturn. so case with unwind exception across KiUserCallbackDispatcher - is wrong by design and must not work. who in this case call ZwCallbackReturn ? if it will be called from __finally handler in KiUserCallbackDispatcher - this break unwind procedure ( ZwCallbackReturn how i say never return, but like long jump - move us to another place with another stack pointer and all registers). even if you manually call ZwCallbackReturn from catch - where you will be after this call ?

so need handle exception before KiUserCallbackDispatcher SEH handler or how minimum have __finally blocks, if need de-allocate some resources.

KiUserCallbackDispatcher used SEH handler for call ZwCallbackReturn even if exception will be in callback. however behavior of this SEH handler can be affected by undocumented flag in RTL_USER_PROCESS_PARAMETERS.Flags :

pseudo-code:

void KiUserCallbackDispatcher(...)
{
    __try {

        //...
    } __except(KiUserCallbackExceptionFilter(GetExceptionInformation())) {
KiUserCallbackDispatcherContinue:
        ZwCallbackReturn(0, 0, 0);
    }
}

void LdrpLogFatalUserCallbackException(PEXCEPTION_RECORD ExceptionRecord, PCONTEXT ContextRecord);

int KiUserCallbackExceptionFilter(PEXCEPTION_POINTERS pep)
{
    if ( NtCurrentTeb()->ProcessEnvironmentBlock->ProcessParameters->Flags & 0x80000)
    {
        return EXCEPTION_EXECUTE_HANDLER;
    }

    LdrpLogFatalUserCallbackException(pep->ExceptionRecord, pep->ContextRecord);

    return EXCEPTION_CONTINUE_EXECUTION;
}

if this flag (0x80000) not set (by default) - LdrpLogFatalUserCallbackException called. it internally call UnhandledExceptionFilter and if it not return EXCEPTION_CONTINUE_EXECUTION - called ZwRaiseException with STATUS_FATAL_USER_CALLBACK_EXCEPTION and FirstChance = FALSE (this mean this exception not passed to application - only for debugger as last chance exception)

if we set this flag - EXCEPTION_EXECUTE_HANDLER will be returned from filter - RtlUnwindEx will be called (with TargetIp = KiUserCallbackDispatcherContinue) - as result __finally handlers will be called and at the end ZwCallbackReturn(0, 0, 0);

in any case exception will be not passed to SEH in function which is higher in stack, than KiUserCallbackDispatcher (because here exception will be handled)

so we need handle exception in stack below KiUserCallbackDispatcher or set flags to 0x80000 - in this case, if we not handle exception before KiUserCallbackDispatcher - our __finally blocks (if exist) will be executed before ZwCallbackReturn which finish callback.

SetProcessUserModeExceptionPolicy not exported in recent windows versions (begin from win8), but yearly code of this api was next(exactly code):

#define PROCESS_CALLBACK_FILTER_ENABLED     0x1

BOOL WINAPI SetProcessUserModeExceptionPolicy(DWORD dwFlags)
{
    PLONG pFlags = (PLONG)&NtCurrentTeb()->ProcessEnvironmentBlock->ProcessParameters->Flags;
    if (dwFlags & PROCESS_CALLBACK_FILTER_ENABLED)
    {
        _bittestandset(pFlags, 19); // |= 0x80000
    }
    else
    {
        _bittestandreset(pFlags, 19); // &= ~0x80000
    }

    return TRUE;
}

test:

WNDPROC oldproc;

LRESULT CALLBACK newproc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
{
    SetWindowLongPtrW(hwnd, GWLP_WNDPROC, (LONG_PTR)oldproc);
    __try {
        *(int*)0=0;
        //RaiseException(STATUS_ACCESS_VIOLATION, 0, 0, 0);

    } __finally {
        DbgPrint("in finally\n");
        CallWindowProc(oldproc, hwnd, uMsg, wParam, lParam);
    }
    return 0;
}

void test()
{
    RtlGetCurrentPeb()->ProcessParameters->Flags |= 0x80000;

    if (HWND hwnd = CreateWindowExW(0, WC_EDIT, L"***", WS_OVERLAPPEDWINDOW,
        CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT,
        NULL, NULL, NULL, NULL))
    {
        oldproc = (WNDPROC)SetWindowLongPtr(hwnd, GWLP_WNDPROC, (LONG_PTR)newproc);
        __try {
            ShowWindow(hwnd, SW_SHOW);
        }__except(EXCEPTION_EXECUTE_HANDLER){
            DbgPrint("no sense. never will be called\n");
        }

        MSG msg;
        while (0 < GetMessage(&msg, hwnd, 0, 0))
        {
            DispatchMessage(&msg);
        }
    }
}

try comment or un-comment RtlGetCurrentPeb()->ProcessParameters->Flags |= 0x80000; line (or NtCurrentTeb()->ProcessEnvironmentBlock->ProcessParameters->Flags & 0x80000 which is the same) and compare effect.

anyway top SEH filter (before call wndproc) never will be called

As always, none of this is guaranteed to work, or even documented. This needs to be explicitly noted in this answer, that currently reads, as if this were a supported scenario. There is a reason, why Microsoft have been making it increasingly harder, to pass exceptions across foreign stack frames. C++ code should probably use C++11 facilities (like [std::rethrow_exception](http://en.cppreference.com/w/cpp/error/rethrow_exception)), where possible, to transfer exception objects across foreign stack frames. — IInspectable, Mar 20 '18 at 11:49
@IInspectable - what question is that and the answer. question at begin touch almost undocumented topics. if want be in documentation limits - no answer at all. or answer - impossible. really catch exception in `_tmain` anyway impossible. but partially possible change behavior of exception handler. all what i try do here - some explain how it internal worked. and this not related to c++ exception (which is shell over windows exception). main - exception in callback of course passed to user mode as usual, but handled in `KiUserCallbackDispatcher` *SEH* — RbMm, Mar 20 '18 at 11:56
There is no such thing as *"almost undocumented"*, in the same way as there is no such thing as *"almost pregnant"*. The OP has linked to an official KB entry, and uses what *is* documented there. Your answer presents a solution, that is based off of solely undocumented implementation details. I proposed an alternative when using C++ exceptions (like the OP), because it already provides facilities to pass exception objects by value. If you want to implement a similar scheme for SEH exceptions, I'm sure that is possible, too. But the OP is using C++ exceptions. — IInspectable, Mar 20 '18 at 12:07
@IInspectable - i only try explain how it internal works. and i can not understand - how do you know that this is not documented - i did not write explicitly about it. strange — RbMm, Mar 20 '18 at 12:10
anyway - code of "documented" `SetProcessUserModeExceptionPolicy` is exactly - taken from win7. if somebody want - can get this code and test, how exception route in callback will change after set flag. or direct call `RtlGetCurrentPeb()->ProcessParameters->Flags |= 0x80000;`. maximum what we can achieve with this - avoid fatal error and, if need, execute `__finally` handlers, for free some resources — RbMm, Mar 20 '18 at 12:19
`SetProcessUserModeExceptionPolicy` is not available in any of the systems, the OP is asking about. It also appears, that you did not understand, what the OP is trying to do: They want to handle C++ exceptions raised in callbacks at the call site. At any rate, giving feedback on your submissions is discarded as a general rule. Any opinion is wrong, unless it is yours. I'll just stop wasting my time here. — IInspectable, Mar 20 '18 at 12:24
@IInspectable - *SetProcessUserModeExceptionPolicy is not available in any of the systems* - i wrote about this in answer. but i paste it actual code from win7 - very simply implement it yourself. or `RtlGetCurrentPeb()->ProcessParameters->Flags |= 0x80000` - the same. if you can read. and op not ask about only `c++` exception. question was about exception in general. my answer is informational. — RbMm, Mar 20 '18 at 12:27

Is there a way to get DisableUserModeCallbackFilter to work in Windows 10?

1 Answers1