13

I need to test a software that treats some mouse movements as "gestures". For such a task I need to emulate mouse movement from point A to point B, not in straight line, but as a real mouse moves - with curves, a bit of jaggedyness etc. Is there any available solution (algorithm/code itself, not a library/exe) that I can use?

Of course I can write some simple sinusoidal math by myself, but this would be a very crude emulation of a human hand leading a mouse. Perhaps such a task has been solved already numerous times, and I can just borrow an existing code? :)

ire_and_curses
  • 68,372
  • 23
  • 116
  • 141
grigoryvp
  • 40,413
  • 64
  • 174
  • 277
  • I would guess with now in 2018 you can learn how to use [Tensorflow](https://www.tensorflow.org), using *machine-learning* with real input of human mouse movement so it could detect it later, that would be the ideal solution. – vsync May 18 '18 at 16:07
  • I'm working on a Tensorflow project to make it happen, have you succeeded to do something? – Hossein Alipour Mar 15 '22 at 00:56

5 Answers5

11

Take a look at the Mouse.simba file which is part of the SRL Framework, a macro program designed to work with online games like Runescape. Runescape has extensive macro detection capabilities so all parts of the SRL Framework have been developed to look as human as possible - the mouse functions in particular.

The code is in Pascal but should be quite easy to read. Look at the MMouse() procedure by BenLand100 it moves very realistically doing advanced movements like loops and overshooting its target as well as continuously varying accelerations and directions (also now has laptop touchpad type movements). If you want to test it you'll need to download SIMBA and enable the SRL-include.

Callum Rogers
  • 15,630
  • 17
  • 67
  • 90
  • Is Pascal case sensitive or not? :) – grigoryvp Jan 09 '10 at 19:52
  • 2
    No, it is not case sensitive. The standard is to capitalise all the words in procedures/function names (like ThisIsAProcedure and AndSoIsThis) and to use thisType ofCapitalisation for variables. Many of the functions it uses relates to the rest of the framework or are ones built into SCAR like GetMousePos. – Callum Rogers Jan 09 '10 at 21:53
6

For anyone in the future: I developed a library for Java, that does exactly what OP is asking. The noise/jaggedness in movement, sinusoidal arcs, overshooting the position a bit, etc. Plus the library is written with extension and configuration possibilities in mind, so anyone can fine tune it, if the default solution is not matching the case. Available from Maven Central now.

https://github.com/JoonasVali/NaturalMouseMotion

Joonas Vali
  • 727
  • 6
  • 11
5

How about recording some real gestures and making a way to play them back? That's going to be as real as anything you can synthesize, and it would be repeatable (which is nice for testing).

Adrian McCarthy
  • 45,555
  • 16
  • 123
  • 175
  • It's a good way, but it will ensure only that program in question operates on some predefined movements. And i want to feed it a million of different movements to test :). – grigoryvp Jan 09 '10 at 17:34
  • Then perhaps you should start with a few recorded paths, and apply fuzz testing to generate millions of variants. – Adrian McCarthy Jan 11 '10 at 15:56
2

Create a simple test app which shows a random positioned dot every two seconds. Follow the dot yourself and record your natural mouse movements.

Viktor Sehr
  • 12,825
  • 5
  • 58
  • 90
2

One way to capture this behaviour could be to define an invisible "grid" on the screen, where you track which squares the mouse moves through on its journey.

If this part works and records properly, there's no need to bombard it with tests, since you know that each grid square does its job. Interpreting the gesture is also made easier with this.

Tor Valamo
  • 33,261
  • 11
  • 73
  • 81