GetAsyncKeyState and VirtualKeys/special characters using JNA (JAVA)

Question

I am working on a two-way private chat that will work in a full screen game.

This is required to let the user to type into a semi-transparent textbox at the top of the screen even when it doesn't have focus.

Using the following code, I can detect ALL physical keys, but have a tough time with virtual keys.

SHIFT is detected.

2 is detected.

However Shift + 2 are detected both as separate keys (Even though [SHIFT+2] gives @ on my keyboard). IE: The program outputs both SHIFT, and 2, but not what they produce: @.

The problem is, how will I convert to a character depending on the keyboard? For example:

• On a UK Keyboard, SHIFT+2 will give me " (quotes).
• On a US keyboard, SHIFT +2 will give me @.

How can I convert to a specific character depending on the keyboard?

Here is the code so far:

static interface User32 extends Library {
    public static User32 INSTANCE = (User32) Native.loadLibrary("User32", User32.class);

    short GetAsyncKeyState(int key);
    short GetKeyState(int key);

    IntByReference GetKeyboardLayout(int dwLayout);
    int MapVirtualKeyExW (int uCode, int nMapType, IntByReference dwhkl);

    boolean GetKeyboardState(byte[] lpKeyState);

    int ToUnicodeEx(int wVirtKey, int wScanCode, byte[] lpKeyState, char[] pwszBuff, int cchBuff, int wFlags, IntByReference dwhkl);

}



public static void main(String[] args)  {   
    long currTime = System.currentTimeMillis();

    while (System.currentTimeMillis() < currTime + 20000)
    {
        for (int key = 1; key < 256; key++)
            {
                if (isKeyPressed(key)) 
                    getKeyType(key);
            }
    }
}



private static boolean isKeyPressed(int key)
{
    return User32.INSTANCE.GetAsyncKeyState(key) == -32767;
}



private static void getKeyType(int key)
{

    boolean isDownShift = (User32.INSTANCE.GetKeyState(VK_SHIFT) & 0x80) == 0x80;
    boolean isDownCapsLock = (User32.INSTANCE.GetKeyState(VK_CAPS)) != 0;


    byte[] keystate = new byte[256];
    User32.INSTANCE.GetKeyboardState(keystate); 


    IntByReference keyblayoutID = User32.INSTANCE.GetKeyboardLayout(0);
    int ScanCode  = User32.INSTANCE.MapVirtualKeyExW(key, MAPVK_VK_TO_VSC, keyblayoutID);






    char[] buff = new char[10];

    int bufflen = buff.length;
    int ret = User32.INSTANCE.ToUnicodeEx(key, ScanCode, keystate, buff, bufflen, 0, keyblayoutID);


    switch (ret)
    {
        case -1: 
            System.out.println("Error");
        break;

        case 0:  // no translation

        break;

        default: 
        System.out.println("output=" + String.valueOf(buff).substring(0, ret));
    }




}

It works fine and outputs the keys pressed, but doesn't work with Shift + combinations. I realize that I could do a "Switch" and change Shift+3 to "£", but this will not work with different keyboards.

Answer 1

Try to use JIntelliType library instead. Its much simplier to use than JNA and it should be able to do SHIFT + key (MOD_SHIFT). The only problem you can have is detecting 3, but thats easy to solve (for example by KeyListener printing code of the key).

Answer 2

GetKeyboardState has some issues, but GetAsyncKeyState seem to work just fine.

Here complete working example of Console Application that reads keyboard state from any window. Tested with 2 non en-us keyboard layouts on Windows 7.

Handles everything =) and in particular the SHIFT+ combinations (i.e. SHIFT+3 will be translated into correct character for the current keyboard layout)

P.S. David, thanx to your code example I finaly figured out the correct parameters for MapVirtualKeyExW and ToUnicodeEx functions :)

P.P.S. The code is in C#, but I guess it can be easily ported to Java (since when I read your code I mistakenly assumed it's C# and only much later noticed "JAVA" in the question title )

using System;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using System.Windows.Forms;

namespace KeyboardInputTest
{
    class Program
    {
        static void Main(string[] args)
        {
            new KeyboardTestClass().RunTest();
        }
    }

    public class KeyboardTestClass
    {
        public void RunTest()
        {
            while (true)
            {
                string keyString = string.Empty;
                if (ReadKeyboardInput(ref keyString) && keyString.Length > 0)
                {
                    Console.WriteLine(string.Format("Pressed: {0}", keyString));
                }
                Thread.Sleep(10);
            }
        }

        public bool ReadKeyboardInput(ref string res)
        {
            var hwnd = WinAPI.GetForegroundWindow();
            var pid = WinAPI.GetWindowThreadProcessId(hwnd, IntPtr.Zero);
            var keyboardLayoutHandle = WinAPI.GetKeyboardLayout(pid);

            foreach (var key in (Keys[])Enum.GetValues(typeof(Keys)))
            {
                if (Keyboard.GetAsyncKeyState(key) == -32767)
                {
                    switch (key)
                    {
                        // handle exceptional cases
                        case Keys.Enter:
                        case Keys.LineFeed:
                            res = string.Empty;
                            return false;
                    }
                    res = ConvertVirtualKeyToUnicode(key, keyboardLayoutHandle, Keyboard.ShiftKey);
                    return true;
                }
            }
            return false;
        }

        public string ConvertVirtualKeyToUnicode(Keys key, IntPtr keyboardLayoutHandle, bool shiftPressed)
        {
            var scanCodeEx = Keyboard.MapVirtualKeyExW(key, VirtualKeyMapType.ToVScanCodeEx, keyboardLayoutHandle);
            if (scanCodeEx > 0)
            {
                byte[] lpKeyState = new byte[256];
                if (shiftPressed)
                {
                    lpKeyState[(int)Keys.ShiftKey] = 0x80;
                    lpKeyState[(int)Keys.LShiftKey] = 0x80;
                }
                var sb = new StringBuilder(5);
                var rc = Keyboard.ToUnicodeEx(key, scanCodeEx, lpKeyState, sb, sb.Capacity, 0, keyboardLayoutHandle);
                if (rc > 0)
                {
                    return sb.ToString();
                }
                else
                {
                    // It's a dead key; let's flush out whats stored in the keyboard state.
                    rc = Keyboard.ToUnicodeEx(key, scanCodeEx, lpKeyState, sb, sb.Capacity, 0, keyboardLayoutHandle);
                    return string.Empty;
                }
            }
            return string.Empty;
        }
    }

    // Win API Imports:
    public enum VirtualKeyMapType : int
    {
        ToChar = 2,
        ToVScanCode = 0,
        ToVScanCodeEx = 4
    }
    public static class Keyboard
    {
        public static bool ShiftKey
        {
            get
            {
                return Convert.ToBoolean((int)GetAsyncKeyState(Keys.ShiftKey) & 32768);
            }
        }

        [DllImport("User32.dll")]
        public static extern short GetAsyncKeyState(Keys vKey);

        [DllImport("user32.dll", CharSet = CharSet.Unicode, EntryPoint = "MapVirtualKeyExW", ExactSpelling = true)]
        public static extern uint MapVirtualKeyExW(Keys uCode, VirtualKeyMapType uMapType, IntPtr dwKeyboardLayoutHandle);

        [DllImport("user32.dll", CharSet = CharSet.Unicode, ExactSpelling = true)]
        public static extern int ToUnicodeEx(Keys wVirtKey, uint wScanCode, byte[] lpKeyState, StringBuilder pwszBuff, int cchBuff, uint wFlags, IntPtr dwKeyboardLayoutHandle);
    }

    public class WinAPI
    {
        [DllImport("user32.dll")]
        public static extern IntPtr GetForegroundWindow();

        [DllImport("user32")]
        public static extern int GetWindowThreadProcessId(IntPtr hwnd, IntPtr lpdwProcessId);

        [DllImport("user32")]
        public static extern IntPtr GetKeyboardLayout(int dwLayout);
    }
}

Answer 3

I got it. After many, many, many hours of searching, I managed to create a method that converts the combination to what it should be on the current keyboard layout. It doesn't deal with dead-keys (such as accents), but it catches all the [SHIFT+Combinations] that I need it to catch.

To use it, call it as follows:

getCharacter(int vkCode, boolean shiftKeyPressed);

So, watch this magic. If I want to get what SHIFT+3 will give me on my keyboard (£), I use:

getCharacter(KeyEvent.VK_3, true);

Here is the code:

public static char getCharacter(int vkCode, boolean shiftKeyPressed)
{

    byte[] keyStates = new byte[256]; //Create a keyboard map of 256 keys

    if (shiftKeyPressed)
    {
        keyStates[16]=-127; //Emulate the shift key being held down
        keyStates[160]=-128; //This needs to be set as well
    }

    IntByReference keyblayoutID = User32.INSTANCE.GetKeyboardLayout(0); //Load local keyboard layout

    int ScanCode  = User32.INSTANCE.MapVirtualKeyExW(vkCode, MAPVK_VK_TO_VSC, keyblayoutID); //Get the scancode

    char[] buff = new char[1];

    int ret = User32.INSTANCE.ToUnicodeEx(vkCode, ScanCode, keyStates, buff, 1, 0, _currentInputLocaleIdentifier);

    switch (ret)
    {
    case -1: //Error
        return (char) -1;

    case 0:  //No Translation
        return (char) 0;

    default: //Returning key...
        return buff[0];
    }
}

Here are the declarations:

final static int MAPVK_VK_TO_VSC = 0;
static IntByReference _currentInputLocaleIdentifier; 

static interface User32 extends Library {

    public static User32 INSTANCE = (User32) Native.loadLibrary("User32", User32.class);


    IntByReference GetKeyboardLayout(int dwLayout);
    int MapVirtualKeyExW (int uCode, int nMapType, IntByReference dwhkl);

    boolean GetKeyboardState(byte[] lpKeyState);

    int ToUnicodeEx(int wVirtKey, int wScanCode, byte[] lpKeyState, char[] pwszBuff, int cchBuff, int wFlags, IntByReference dwhkl);

}

A big thank you to BrendanMcK, who helped me to get to this solution.