-1

I haven't find anything relevant in Google or any Microsoft site about it so I decided to ask a question here.

Everybody knows that in Win-based OS there is a virtual keyboard. I also know that *nix based OS, have it too. So, the question is about:

HOW DOES IT WORK INSIDE?

I mean, let's have an example that I opened on screen keyboard in Windows 10. What's the actual difference between:

  1. input via hardware keyboard: when I'm using it, like I press X button
  2. ..and using a virtual keyboard, when I press the same button

Imagine, I have an admin access to terminal/computer, is there any option to track/distinguish that in the second time I pressed button not on hardware keyboard, but on-screen (by mouse clicking) version of it?

And there are also many different software, like AutoIt (yes, it's a language, but it's relevant to this example) that emulating pressing the X button. How does they work in Win-based OS? Do they "in-common" with default on-screen keyboard and using the same driver/WinAPI or there is a difference between them?

And the second case, between:

  1. default on-screen keyboard
  2. compilated AutoIt script
  3. ..any other software that emulating press X button

I guess the only way to find out "how exactly button was pressed" is to check current processes list via taskmgr and find out have anything been launched or not. Or I'm totally wrong here, and missing something?

THE SCOPE

I have written a node.js script which emulates button pressing behaviour in windows app.

TL:DR business logic short => open notepad.exe and type `Hello world`

And could someone give me any advice/recommend any powershell/bat script (or any other solution) with demonstration of Get­Async­Key­State check behavior? With which I could easily check my own node.js script (not by functional of it, but by triggering press the X button event) I found an answer for node.js case here: Detecting Key Presses Across Applications in Powershell

Javan
  • 189
  • 1
  • 7
AlexZeDim
  • 3,520
  • 2
  • 28
  • 64
  • 3
    You'll have to call Microsoft to get a 100% correct answer since only they have access to the source code. But osk.exe has an import for the [olden standard way](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput) of doing this, so likely what it uses. The only way an app could ever see the difference between injected input and normal keyboard input is by using SetWindowsHookEx to set a WH_KEYBOARD_LL hook, the callback will have the LLKHF_INJECTED flag set. – Hans Passant Sep 07 '19 at 18:47
  • @HansPassant, well ofc they are, because software is proprietary, but I guess, that *for the basics* we don't need to analyse the whole source code. And you already mentioned `WH_KEYBOARD_LL ` hook, so it's very helpful for me. – AlexZeDim Sep 07 '19 at 19:10

1 Answers1

1

SendInput is the preferred method to generate user input in software. The Windows on-screen keyboard probably uses it for everything except Ctrl+Alt+Delete which I believe has some kind of special handling. The on-screen keyboard is only able to generate Ctrl+Alt+Delete in certain configurations.

Software-generated input is merged with normal hardware input in the RIT (Raw Input Thread) in the kernel.

A low-level keyboard hook can detect software-generated input.

Anders
  • 97,548
  • 12
  • 110
  • 164
  • So, if I'm understood it correctly, on-screen keyboard uses `SendInput` and software-injected input *uses* memory of the process (for the lack of better words) that it's hooked to? Like, if I open `notepad.exe` and emulate via software `press X button`, software will inject directly to `notepad.exe` process? – AlexZeDim Sep 07 '19 at 19:16
  • 1
    Both when you press your keyboard OR call SendInput, the input is processed in the kernel. In most cases the input makes its way to the active window where it is sent to this window as WM_KEYDOWN/UP messages. You cannot use memory to fake input but you can send fake WM_KEYDOWN/UP messages with SendMessage (not recommended). – Anders Sep 07 '19 at 19:28
  • 1
    Raymond Chen has a [blog article](https://devblogs.microsoft.com/oldnewthing/20140213-00/?p=1773) that shows a diagram of how physical input and simulated input are related to each other, and the flow of input to a target window – Remy Lebeau Sep 07 '19 at 19:34
  • @RemyLebeau this is extremely helpful, and exactly "what am I looking at" (unfortunately I search for: `how virtual keyboard works`) So this article represents behaviour of `SendInput` for all WindowsNT architecture, am I right? – AlexZeDim Sep 07 '19 at 19:55
  • 1
    @AlexZeDim Almost. `SendInput` was added in Win98 or 2000. Older versions have a older function (`keybd_event`) that does not properly synchronize the input but I assume you don't care about those versions? – Anders Sep 07 '19 at 19:59
  • @Anders Yeah, actually I'm interested in Win8+ retail and server versions of it. So it's 100% relevant for me. Also I have another relevant question, but I don't like to create another Q on SW. So I edit this question (in 5 mins) at the `end stage` and if you find time/knowledge for it, I'll be appreciate if you answer. (Right after if, I mark this question as `solved`) – AlexZeDim Sep 07 '19 at 20:05
  • 1
    GetAsyncKeyState is not the best way to detect input. A real Windows application handles WM_KEYDOWN/UP/CHAR messages. – Anders Sep 07 '19 at 21:22