Sorry about the long post! I have some explaining to do, I'm afraid... I have an application (in Unity3d, but for windows) that doesn't use WinForms, so I cannot use the Cursor class which is part of System.Windows.Forms namespace. But all the same, I'd like to be able to set the current shape of the cursor.
After much research and much googling, I've found that this is possible using P/Invoke calls to certain methods in user32.dll. I got it (sort of) working by:
Find a handle to the window using:
[DllImport("user32.dll")]
public static extern IntPtr FindWindow(string lpClassName, string lpWindowName);
And supply null for the classname and the title of the window for the windowname. Then use the returned IntPtr to call this:
[DllImport("user32.dll")]
public static extern IntPtr SetClassLong(IntPtr hwnd, int index, IntPtr dwNewLong);
And supply "-4" for the index in order to target the pointer to the WindowProc that handles all the lowlevel window messages, and then use:
System.Runtime.InteropServices.Marshal.GetFunctionPointerForDelegate()
To generate a native pointer to a new WndProc delegate which I implement in my managed code. Its signature looks like this:
public long WndProc(IntPtr hwnd, uint msg, uint wparam, int lparam);
In summary, I'm basically overriding the native window procedure with a managed callback, and it then becomes my responsibility to handle all the window messages. But I'm not interested in rewriting the entire implementation for the standard window proc, I just want to be able to control the way it draws its mouse cursor shape when the cursor is over the window. So, to do so, I can call the default window proc with this function:
[DllImport("user32.dll")]
public static extern long DefWindowProc(IntPtr hwnd, uint msg, uint wparam, int lparam);
And then immediately afterwards, set the cursor to something else using:
[DllImport("user32.dll", CharSet = CharSet.Auto, ExactSpelling = true)]
private static extern IntPtr SetCursor(IntPtr hCursor);
This sort of works. :) I say "sort of", because I'm seeing a bit of flickering that occurs because the default window proc first sets the cursor to the class cursor, and then immediately afterwards, my implementation sets it to the custom cursor I want using SetCursor. This situation can be remedied by setting the class cursor to null. Doing so causes the system to not draw the cursor and expect the application to set the cursor for every window message instead, and that's exactly what I'm doing, so it would stop them fighting over the who gets to set the cursor and eliminate the flickering. This is documented on MSDN at:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms648393%28v=vs.85%29.aspx
Where they explain that one must set the class cursor to null. But that's where my knowledge falls short. :( The class cursor is set with SetClassLong as explained in the article above.
But the datatype for the new value of the cursor is IntPtr. This datatype is not nullable, the compiler complains if I pass in null, and passing in IntPtr.zero doesn't work. So how do I accomplish what the article advices? How do I set the class cursor to null using P/Invoke?
