Getting the instance handle of a window in python (ctypes)

Question

I've been trying to create a simple window using win32api but in python using the ctypes lib.

Now the first thing i need is the programs console windows instance handle.

For getting that i need to get a handle to the console window which is doable by using Kernel.dll GetConsoleWindow() Function. Then i pass that to the GetWindowLongA() Function of the User32.dll library which should return the python equivalent of HINSTANCE in windows.h header of C which should be an int. but all i get is 0 which means it has failed.

The code i currently have is:

windows = windll.LoadLibrary("User32.dll")
kernel  = windll.LoadLibrary("Kernel32.dll")

consoleHandle = kernel.GetConsoleWindow()
print(f"Console HWND : {consoleHandle}")
instanceHandle: ctypes.wintypes.HINSTANCE = windows.GetWindowLongA(consoleHandle, c_int(-6))
print(f"Console HINSTANCE : {instanceHandle}")
print(f"LAST FUNCTION ERROR : {kernel.GetLastError()}")

OUTPUT :

Console HWND : 198610
Console HINSTANCE : 0
LAST FUNCTION ERROR : 1413

now error 1413 is ERROR_INVALID_INDEX (based on System error codes) which is probably caused cause i passed c_int(-6) as the int nIndex argument which must represent the GWL_HINSTANCE macro defined in windows.h (based on this windows doc) but it doesnt work.

What should i do to fix it and why doesnt it work?

Answer 1

Console Windows are not the same as GUI Windows. GetActiveWindow() is what you want, and needs to be run for a GUI Python IDE, not a console window.

Assuming 64-bit Windows and Python, you also need GetWindowLongPtrA() (or W) to retrieve handles.

To ensure you retrieve the last error cause by a Windows API failure using ctypes, use the use_last_error=True parameter and call ctypes.get_last_error() to retrieve it. The error code is only valid after the specific Windows call that failed, ctypes will capture that with this method. Otherwise, you don't know what Windows APIs might be called by the Python implementation between the ctypes API call to the function that failed and the ctypes GetLastError() API call. I got 1400 (invalid window handle) when calling GetWindowLongPtrA() with a console handle.

It's also good practice to fully specify .argtypes and .restype, esp. for 64-bit Windows as the default return type is c_int (32-bit) and handles are 64-bit in that architecture.

Below works for a 64-bit Python IDE:

import ctypes as ct
from ctypes import wintypes as w

GWLP_HINSTANCE = -6

# LONG_PTR isn't defined by wintypes.
# LPARAM is long on 32-bit, longlong on 64-bit,
# which matches LONG_PTR type.
LONG_PTR = w.LPARAM

u32 = ct.WinDLL('user32', use_last_error=True)
GetWindowLongPtrA = u32.GetWindowLongPtrA
GetWindowLongPtrA.argtypes = w.HWND, ct.c_int
GetWindowLongPtrA.restype = LONG_PTR
GetActiveWindow = u32.GetActiveWindow
GetActiveWindow.argtypes = ()
GetActiveWindow.restype = w.HWND

hwnd = GetActiveWindow()
print(f'hwnd = {hwnd:#x}')
instanceHandle = GetWindowLongPtrA(hwnd, GWLP_HINSTANCE)
print(f'instanceHandle = {instanceHandle:#x}')

Output:

hwnd = 0xd044c
instanceHandle = 0x7ff700250000