11

I'm attempting to set up a low-level keyboard hook using P/Invoke in an F# application. The Win32 function SetWindowsHookEx takes a HOOKPROC for its second argument, which I've represented as a delegate of (int * IntPtr * IntPtr) -> IntPtr, similar to how this would be handled in C#. When calling the method, I get a MarshalDirectiveException stating that the delegate parameter cannot be marshaled because

Generic types cannot be marshaled

I'm not sure how generics are involved, as all types are concretely specified. Can anyone shed some light on this? Code follows.

EDIT

This may have to do with the way the F# compiler deals with type signatures - Reflector indicates that the delegate LowLevelKeyboardProc is implemented as a method which accepts one argument of type Tuple<int, IntPtr, IntPtr> - and there would be the un-marshalable generic type. Is there a way around this somehow, or are F# functions simply not capable of being marshaled to native function pointers?

let WH_KEYBOARD_LL = 13

type LowLevelKeyboardProc = delegate of (int * IntPtr * IntPtr) -> IntPtr

[<DllImport("user32.dll")>]
extern IntPtr SetWindowsHookEx(int idhook, LowLevelKeyboardProc proc, IntPtr hMod, UInt32 threadId)

[<DllImport("kernel32.dll")>]
extern IntPtr GetModuleHandle(string lpModuleName)

let SetHook (proc: LowLevelKeyboardProc) =
    use curProc = Process.GetCurrentProcess ()
    use curMod = curProc.MainModule

    SetWindowsHookEx(WH_KEYBOARD_LL, proc, GetModuleHandle(curMod.ModuleName), 0u)
ildjarn
  • 62,044
  • 9
  • 127
  • 211
Ben
  • 6,023
  • 1
  • 25
  • 40
  • +1 for trying to p/invoke from F# and *passing a delegate*. I honestly don't know if that can even be done. – Frédéric Hamidi Apr 15 '11 at 22:45
  • I think the result type should simply be `int`, not `IntPtr`. But I guess that will not make a difference with the error message...; suggestion: add the code that calls `SetHook`, maybe there is a problem there? – wmeyer Apr 15 '11 at 23:26
  • @wmeyer : A return type of `int` rather than `nativeint` would fail on 64-bit platforms. – ildjarn Apr 15 '11 at 23:39
  • @Frédéric Hamidi : This is standard for passing callbacks to native code in all .NET languages; nothing special about using F# for this. – ildjarn Apr 15 '11 at 23:43
  • @ildjarn, I was wary of F# functions used as delegates for some reason, but your mentioning of curried and tupled forms clarified that. Thanks :) – Frédéric Hamidi Apr 15 '11 at 23:49
  • 2
    @Frédéric Hamidi : I'll admit, it's not obvious at first. `delegate of (int * int) -> int` defines a unary delegate taking an actual `Tuple` whereas `delegate of int * int -> int` defines a binary function taking two `int`s. But, the latter must be used with a curried function even though it *appears* to use tuple syntax. Anyway, the non-obvious semantics of the former version is what was causing Ben's error, as `Tuple` is quite clearly a generic. – ildjarn Apr 15 '11 at 23:53
  • @ildjam Non-obvious semantics indeed! Thanks for clearing this up; the apparent inability to create a "proper" delegate in F# was really bothering me. – Ben Apr 16 '11 at 02:53

2 Answers2

16

Your LowLevelKeyboardProc definition is wrong. Change from

type LowLevelKeyboardProc = delegate of (int * IntPtr * IntPtr) -> IntPtr

to

type LowLevelKeyboardProc = delegate of int * IntPtr * IntPtr -> IntPtr

or better yet

type LowLevelKeyboardProc = delegate of int * nativeint * nativeint -> nativeint

or better yet

[<StructLayout(LayoutKind.Sequential)>]
type KBDLLHOOKSTRUCT =
    val vkCode      : uint32
    val scanCode    : uint32
    val flags       : uint32
    val time        : uint32
    val dwExtraInfo : nativeint

type LowLevelKeyboardProc =
    delegate of int * nativeint * KBDLLHOOKSTRUCT -> nativeint

In all of the above cases, proc will need to use curried form rather than tupled form.

Also note that you should add SetLastError = true to all of the externed functions whose documentation says to call GetLastError upon failure (which is the case for GetModuleHandle, SetWindowsHookEx, and UnhookWindowsHookEx). That way if any fail (and you should be checking the return values...), you can simply raise a Win32Exception or call Marshal.GetLastWin32Error to get proper diagnostics.

EDIT: Just for the sake of clarity, here are all the P/Invoke signatures I successfully tested locally:

[<Literal>]
let WH_KEYBOARD_LL = 13

[<StructLayout(LayoutKind.Sequential)>]
type KBDLLHOOKSTRUCT =
    val vkCode      : uint32
    val scanCode    : uint32
    val flags       : uint32
    val time        : uint32
    val dwExtraInfo : nativeint

type LowLevelKeyboardProc = delegate of int * nativeint * KBDLLHOOKSTRUCT -> nativeint

[<DllImport("kernel32.dll")>]
extern uint32 GetCurrentThreadId()

[<DllImport("kernel32.dll", SetLastError = true)>]
extern nativeint GetModuleHandle(string lpModuleName)

[<DllImport("user32.dll", SetLastError = true)>]
extern bool UnhookWindowsHookEx(nativeint hhk)

[<DllImport("user32.dll", SetLastError = true)>]
extern nativeint SetWindowsHookEx(int idhook, LowLevelKeyboardProc proc, nativeint hMod, uint32 threadId)

Also note that this would work equally as well, if you prefer value semantics for KBDLLHOOKSTRUCT:

[<Struct; StructLayout(LayoutKind.Sequential)>]
type KBDLLHOOKSTRUCT =
    val vkCode      : uint32
    val scanCode    : uint32
    val flags       : uint32
    val time        : uint32
    val dwExtraInfo : nativeint

type LowLevelKeyboardProc = delegate of int * nativeint * byref<KBDLLHOOKSTRUCT> -> nativeint
ildjarn
  • 62,044
  • 9
  • 127
  • 211
  • how do you usually test your P/Invokes? – GregC Apr 16 '11 at 00:53
  • 1
    @GregC : Depends on the level of complexity. In this case, I wrote a small C++ console app to determine expected behavior then verified that the F# app had the same. In more complex cases I usually write a C++ .dll that exports functions with the same signatures of those I ultimately wish to call, and have those functions just blindly forward data back and forth to the underlying WinAPI functions, logging as it does so. Then I can look over the logging output to make sure the marshaling behavior is sane. – ildjarn Apr 16 '11 at 01:31
1

Have you tried to use managed C++ with this. It can make a lot of the translation pretty seamless. You won't need P/Invoke then.

EDIT: I'd like to note one fairly important thing: compiler will do more type-checking for you. I am sure you love your type-checking, since you use F# for the rest of the app (hopefully).

GregC
  • 7,737
  • 2
  • 53
  • 67