3

I know about the input tap x y shell command, however, I'm trying to understand how to perform a click using the sendevent command. I been able to achieve it with the following command:

sendevent /dev/input/event5 3 53 X &&
sendevent /dev/input/event5 3 54 Y &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0

Where X and Y is the position that will be clicked, I'm testing it on the android emulator BlueStacks 5 which the Display Resolution set to 1920x1080.

The code is working and the click is fired, however, I couldn't understand how to convert the position where I want to be clicked to the sendevent XY position.

If I send using ADB:

sendevent /dev/input/event5 3 53 2000 &&
sendevent /dev/input/event5 3 54 2000 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0

It clicks somewhere around x75 y75, how this calc is done? i mean screen xy -> sendevent xy?

How to replicate:

  • First enable BlueStacks 5 adb in the window: Settings -> Advanced -> Android debug bridge
  • Open a cmd window and run cd C:\Program Files\BlueStacks_nxt assuming BlueStacks where installed in the default path.

Execute the commands:

  • hd-adb.exe connect 127.0.0.1:X where X is the port shown in the window where you enabled the ADB.
  • hd-adb.exe -s 127.0.0.1:X shell

Now we are on the shell, execute a new command: getevent -p and search for:

... /dev/input/event5
  name:     "BlueStacks Virtual Touch"

On my emulator the input event for touch is event5 on yours it can be different, replace it according.

Now you can simulate a click with the code below changing XY to the position where you want to be clicked:

    sendevent /dev/input/event5 3 53 X &&
    sendevent /dev/input/event5 3 54 Y &&
    sendevent /dev/input/event5 0 2 0 &&
    sendevent /dev/input/event5 0 0 0 &&
    sendevent /dev/input/event5 0 2 0 &&
    sendevent /dev/input/event5 0 0 0

I'm trying to figure out how to convert the emulator screen position to the sendevent position.

For example, if you want to perform a click at x200 y200, using sendevent what the value needed?

How to calculate it?

Cesar
  • 41
  • 2
  • 5
  • 16
  • I believe those x and y are absolute screen coordinates (so not just window coordinates, you have to include the toolbar, statusbar, and other decorations). – Gabe Sechan Sep 12 '22 at 02:48
  • @GabeSechan do you think is possible to calculate/convert? i mean any kind of formula – Cesar Sep 12 '22 at 02:57
  • view.getLocationOnScreen() will return the screen coordinates of the upper left corner of the view. THen you can add in whatever offset you want. These will be specific to whatever emulator settings/device settings you have, as density and decorations are not the same globally. – Gabe Sechan Sep 12 '22 at 03:03
  • @GabeSechan when i run `view.getLocationOnScreen()` on shell nothing happens – Cesar Sep 12 '22 at 03:14
  • 1
    It's not a shell command, its a function in the app. There's no way to do it from the shell, because the idea of a View isn't something that goes beyond the app it's in. – Gabe Sechan Sep 12 '22 at 03:31
  • You wrote that the screen resolution is 1920x1080 but you send screen coordinates of 2000, 2000. Is it a typo? – TDG Sep 14 '22 at 15:07
  • 1
    @TDG no, to click on x1900~ using the ADB on the emulator i need to send `sendevent` with a value higher than `30000` weird. – Cesar Sep 14 '22 at 15:10
  • Indeed weired, did you try it with a physical device? – TDG Sep 14 '22 at 15:40
  • No i dont only with the emulator – Cesar Sep 14 '22 at 15:43

1 Answers1

3

thanks for the very precise instructions to reproduce:

I enabled
Settings -> Advanced -> Input debugging -> Show visual feedback for taps
and
Settings -> Advanced -> Input debugging -> Show pointer location for current touch data

when I hold down click, I can see: X: Y:

I collected these x coordinate points, my max X: is 1600.0 so my width is 1600.0

(8000, 390.6)
(16000, 781.2)
(32000, 1562.5)

then Excel: X Y (scatter) chart, add trendline, click trendline and click big + sign -> Chart Elements -> Trendline : (tick that) and ▶, More Options... -> Trendline Options -> Display Equation on chart
, click on the formula, then Label Options -> Category: Number, Decimal Places: 10
y = 0.0488294643x - 0.0500000000

1600 = 0.0488294643x - 0.0500000000
32768.1252075501 = X

I round to 32768 because 32768 is a magic number, close to Int16's 32767
edit: after Nathan's comment, it's 32767 (32768 doesn't even move the cursor)

so the formula is : (W: width, H: height)

32767*X/W
32767*Y/H

for your (X=200, Y=200, W=1980, H=1080)

32767*200/1920
32767*200/1080

3413.22916666667
6067.96296296296

I used this code to test: it doesn't do a tap, it holds down without releasing

sendevent /dev/input/event5 3 57 0
sendevent /dev/input/event5 3 53 3413.22916666667
sendevent /dev/input/event5 3 54 6067.96296296296
sendevent /dev/input/event5 3 48 5
sendevent /dev/input/event5 3 58 50
sendevent /dev/input/event5 0 2 0
sendevent /dev/input/event5 0 0 0

https://ktnr74.blogspot.com/2013/06/emulating-touchscreen-interaction-with.html#:~:text=ABS_MT_TRACKING_ID%20(57)%20%2D%20ID,end%20of%20report

Mr. Doge
  • 796
  • 5
  • 11
  • 1
    Thank you! I didn't understand how you get these values `0.0488294643x - 0.0500000000` – Cesar Sep 15 '22 at 23:20
  • I omitted a step by accident but : Excel: `+` -> Chart Elements -> Trendline : (tick that) and ▶, More Options... -> Trendline Options -> Display Equation on chart
    basically it computes the line of best fit and it gives the slope, I could very well have done `((781.2 - 390.6) / (16000 - 8000))` to get the slope since the points are almost perfectly aligned, but I didn't know that     – Mr. Doge Sep 15 '22 at 23:40
  • so that was just looking if there was a mathematical relationship between these points, it's `y = mx` ? or as Gabe said: "you have to include the toolbar, statusbar, and other decorations" : `y = mx + b` ? it's easier with a formula than to manually figure out by code the size of the window ? : I'm thinking of : – Mr. Doge Sep 15 '22 at 23:45
  • 1
    The toolbar has a fixed height of 32px, it is not possible to click on it using `sendevent` because he's on a different window and this window is not related to the android/adb. I'm testing it, looks like the formula is really `32767*x/w` `32767*y/h` `32768` on `sendevent` doesn't even fire the click. Thank you, Doge! – Cesar Sep 16 '22 at 00:58