0

I'm writing a application profiling library that basically hooks Windows APIs and records the parameters and results. I'm trying to come up with a way to generate these hooks in a manner using C++ templates that requires minimal effort to add new hooks. Basically, each one of my hooks look like the following:

BOOL HookCloseHandle(HANDLE h)
{
    BOOL result = RealCloseHandle(h);
    if(g_profiling) record(result, h);
    return result;
}

I want to generalize that via templates so these functions can be generated for any arbitrary Windows API function via decltype, e.g. decltype(CreateFileW). Is this even possible? I've been looking at function_traits in Boost and it seems I'm able to come up with something close:

decltype(&CloseHandle) RealCloseHandle = &::CloseHandle;

template <typename R, typename P1, R(*F)(P1)>
R perform_1(P1 value)
{
    R result = F(value);
    if (profiling) record(result, value);
    return result;
}

template <typename T>
T* call(const T& original)
{
    typedef boost::function_traits<T> traits;
    switch (traits::arity)
    {
        case 1:
            return &perform_1<traits::result_type, traits::arg1_type, RealCloseHandle>;
        // ...
    }
    return nullptr;
};

// setup code
Hook(RealCloseHandle, call<decltype(CloseHandle)>());

Where Hook is provided by the hooking library that replaces a "real" function with my hooked version.

The only thing is, i'm not sure how to remove that CloseHandle template argument that's currently inside the call function. Any ideas?

atanamir
  • 4,833
  • 3
  • 24
  • 20
  • Shouldn't you just use `call(RealCloseHandle)`? – R Sahu Apr 22 '14 at 21:50
  • I need a templated function that is compiled with the call to `RealCloseHandle` (i.e. it's not passed in), not a function that takes a function pointer as its first argument, so `RealCloseHandle` will need to be a template argument somehow. – atanamir Apr 22 '14 at 21:53
  • 1
    here's my two cents: http://coliru.stacked-crooked.com/a/a274bbb6fb995209. The calling syntax is slightly odd, but since it only deals with standard-layout temporaries with no members, it's _trivially_ optimized by the compiler to 0 overhead. – Mooing Duck Apr 22 '14 at 22:06
  • Your solution looks really promising, but check out how it behaves on MSVC++ 2013: `var.cpp(31) : fatal error C1001: An internal error has occurred in the compiler.` :( – atanamir Apr 22 '14 at 22:36
  • @MooingDuck For your solution though, is there a way to get a function pointer to the `()` operator which *matches that of the original function*? Even the calling convention... (that's what the API hooking library needs) – atanamir Apr 22 '14 at 22:49
  • @atanamir: Added a `.get()` which returns the original function pointer, clearer calling syntax, and specializations to handle functions that return `void`: http://coliru.stacked-crooked.com/a/d9a007634bce38d9. Visual Studio _claims_ it can handle this version (http://rise4fun.com/Vcpp/H1N), did not test my prior. (Note that this does _not_ do perfect forwarding, so works _very badly_ with large or complex types, but C APIs doesn't have those so...) – Mooing Duck Apr 22 '14 at 23:04
  • @MooingDuck wow, it's becoming beautiful. However, I think you misunderstood my previous question -- this is hooking an existing binary's calls, so I need to get a function pointer to the generated implementation (the one with `record()` in it); the function needs to be invokable using the same calling convention as the original function in order to be used as a hook. Right now it seems that the `operator ()` will be `__thiscall`, whereas the original `CloseHandle` is `__stdcall`, so it seems they aren't compatible. Is that correct? – atanamir Apr 22 '14 at 23:08
  • @atanamir: Ooooh, that's correct. More relevent, they're member functions. Both are easily fixed: http://coliru.stacked-crooked.com/a/40c132d2c963a98c (I've renamed it `call` since you'll be wanting something easier to type) – Mooing Duck Apr 22 '14 at 23:14
  • ....And this time it actually compiles: http://coliru.stacked-crooked.com/a/3515d42653165348 (Also, a hex upon whoever put the `stdcall` specifiers in a different place than their compeditor!) – Mooing Duck Apr 22 '14 at 23:22
  • Cool! I'll try it on my machine now too. One last question: is it possible to use the calling convention specified in template argument? i.e. `decltype(CloseHandle)` has the calling convention in it... other than that, this is really cool. If you want to put it in as an answer, i can give you the rep for it. – atanamir Apr 22 '14 at 23:25
  • @MooingDuck: please put it in an answer. (P.S. GCC 4.9 and ICC 13 can't digest the code -- only CLang seems to be able to.) – LThode Nov 12 '14 at 20:42
  • @atanamir: Calling conventions are only sometimes inconsistently part of the type, and in a template argument the calling convention is lost. Very sad. – Mooing Duck Nov 12 '14 at 21:54

1 Answers1

1

Well, the basics look like this:

//declare the Wrap type
template<class signature, signature func> struct Wrap;

//template parameters are return type, parameter list, then the actual function
template<class R, class...Ps, R(&func)(Ps...)> 
struct Wrap<R(Ps...), func>
{        
    static R MSVCSTDCALL call(Ps...Vs) GCCSTDCALL
    {
        auto result = func(Vs...);
        if (g_profiling) record_w_ret(Wrap(), result, Vs...);
        return result;
    }
};

//pass the call function of the Wrap object
Hook(CloseHandle, Wrap<void(HANDLE),CloseHandle>::call);

However, this doesn't handle void returns, so we need a specialization for that:

//Wrappers without returns
template<class...Ps, void(&func)(Ps...)> 
struct Wrap<void(Ps...), func>
{        
    static void MSVCSTDCALL call(Ps...Vs) GCCSTDCALL
    {
        func(Vs...);
        if (g_profiling) record_no_ret(Wrap(), Vs...);
    }
};

And to fix the ugly usage syntax:

//easymode wrapper
#define WRAP(X) Wrap<decltype(X),X>::call

Hook(CloseHandle, WRAP(CloseHandle));

However, it has been pointed out to me that using a function pointer as a non-type-template-parameter is illegal and G++ won't take it. I'll work on that. Clang and MSVC accept them fine.

Mooing Duck
  • 64,318
  • 19
  • 100
  • 158