2

I am just about to finish my final degree project. I am creating a system that reads a gesture from the Microsoft Kinect and then it makes some actions. So now I'd like to include some interaction with some famous applications.

For example, I'd like to "tell" the Adobe Reader (that has the focus) to go to the next page or to "tell" the Windows Media Player to stop the music.

Which is the best way to do it? I tried to send a keystroke to the window handler of the application but It didn't work. Then I realized that within a window handler there other several window handler but I couldn't managed to access them.

Thanks in advance!

Oni
  • 798
  • 2
  • 8
  • 17
  • Why not simulate keystrokes since the application has focus? Something like MEDIA_PLAY, MEDIA_PAUSE, KEY_UP and DOWN? – RedX Jul 26 '11 at 14:13
  • @RedX: those keys would only be helpful in the context of media player, I think the OP wanted a more general solution. – Chris Eberle Jul 26 '11 at 14:15
  • @Chris, I meant ARROW_UP and DOWN as the last two. Those can be used to scroll documents for instance. – RedX Jul 26 '11 at 14:17
  • @RedX: Fair enough. See my answer for why that solution isn't all that ideal. – Chris Eberle Jul 26 '11 at 14:20
  • @Chris I did read your post before my first comment. The simulated input is IMHO the best option. I'd just define gestures depending on the focused applicaton. Something like a closed fist on media player is stop, and on dialog button (or any other window) is enter. You could use a config file like many remote control apps do nowadays. – RedX Jul 26 '11 at 14:24
  • Thanks to both of you! Let me explain something. The rules are based on the gesture AND the application that has te focus. For example, the gesture A with application B would do something different from gesture A with application C. I just have tried to do the next: 'HWND x = GetForegroundWindow(); SendMessage(x, WM_VSCROLL, SB_PAGEDOWN, 0);' with Internet Explorer and it does not work. Why? Because the window handler that I have to send the message is not the main window but the controller (Scroll bar in this case) itself. The problem is that I can't access more than one level inside. Thanks! – Oni Jul 26 '11 at 14:35

2 Answers2

1

There are generally only a few "acceptable" solutions to problems like this:

Simulate user input

Programatically send keystrokes and mouse clicks

  • Pros: Compatible with most GUI programs you can think of
  • Cons: Kludgy, not good at handling any kind of unexpected behavior (i.e. error dialogs). Have to know which windows you're expecting to communicate with ahead of time. Dialogs change from one version to the next, so your program often stops working with new releases.

Program API

Communicate directly with the program using calls that it natively understands (play, stop, etc).

  • Pros: Much easier programming experience, cleaner and fewer bugs than simulating input
  • Cons: Very few applications actually expose an API. And for those that do, you're limited to the functionality supported by the API (which may not include everything the program is capable of).

Scripts

Some programs allow scripting languages (VBScript for example)

  • Pros: Same pros as API
  • Cons: Have to somehow programatically get the program to open the script, which may or may not be easy depending on the program.

As for those programs specifically I can't comment. I don't know if they expose any kind of API. I imagine some of them might (media player for example I would think has some of this). Don't quote me on that though. I hope this helps.

Chris Eberle
  • 47,994
  • 12
  • 82
  • 119
  • Thanks Chris! I would like to find the second solution (API) for some of the most famous applications like Windows Media Player, PowerPoint and Adobe Reader but I couldn't find them. On simulating the user input, this is the first one I went for but It is actually more complicate than I first thought. Now I am thinking in how to send HotKeys to a window. Does anyone have a clue? Thanks :) – Oni Jul 27 '11 at 00:01
0

Check out autohotkey

http://www.autohotkey.com/

it is a tool were you can create scripts that controls the mouse/keyboard.

A suggestion is to link a gesture to a autohotkey script. That way, your program can be very configurable and you can test the scripts pretty easily.

Lucian
  • 3,407
  • 2
  • 21
  • 18