6

I've been struggling to use tweening to make mouse movements smooth in Python, I am currently trying to automate some repetitive tasks.

I've tried to use tweening to remove some of the roughness that occurs without smoothing applied, however by doing so I am losing a noticeable amount of accuracy, after all my dy and dx values are getting split by a number I end up with remainders. This could possibly be solved by getting the greatest common factor on both my values (since both dx and dy need to be split by the same number) unfortunately this leads to a too small of a GCD.

Since the mouse cannot move the remainder of a pixel on a screen I end up a with noticeable loss of accuracy.

Question: How to apply tweening on mouse movements, without losing accuracy?

import pytweening
import win32api
import win32con
from time import sleep

dy = [50, 46, 42, 38, 33, 29, 24, 20, 15, 10, 10]
dx = [-35, 6, -55, -43, 0, 17, 29, 38, 42, 42, 38]

while True:

    count = 0

    values = [(pytweening.getPointOnLine(0, 0, x, y, 0.20)) for x, y in zip(dx, dy)]

    while win32api.GetAsyncKeyState(win32con.VK_RBUTTON) and win32api.GetAsyncKeyState(win32con.VK_LBUTTON):

        if count < len(dx):

            for _ in range(5):
                win32api.mouse_event(1, int(values[count][0]), int(values[count][1]), 0, 0)
                sleep(0.134 / 5)

            count += 1
Carson
  • 6,105
  • 2
  • 37
  • 45
Naomi
  • 271
  • 1
  • 3
  • 15
  • As far as I can tell (without being familiar with `pytweening`), you are picking a single tween point between each two of your given points, and aren't even attempting to move to the final x,y coordinates. I believe you should be looping that final parameter too `.getPointOnLine()` between 0 and 1. – jasonharper Dec 19 '19 at 01:01
  • In this example I am splitting the values 5 times `0.20`. Then in my for loop I loop over those two values 5 times. – Naomi Dec 19 '19 at 01:06
  • You are providing the *exact same coordinates* to `.mouse_event()` each of those 5 times - in other words, you are performing no smoothing at all. – jasonharper Dec 19 '19 at 01:09
  • It's something to think about. Can you explain why I see a clear difference when running my code with the above functionality and without? – Naomi Dec 19 '19 at 01:29
  • 1
    Please edit the code in your question to make it a [mre] - so I can paste it into a file and __without adding anything__ run it to see the behaviour you are seeing – DisappointedByUnaccountableMod Dec 24 '19 at 16:14
  • is repetitive tasks must need to move the mouse? (I mean you can click directly) If you need to draw something smooth, you can consider using the ``Bezier curve``. – Carson Dec 31 '19 at 03:11

2 Answers2

5

The fundamental problem here is that you are using relative movement in integer amounts, which will not add up to the total movement you are looking for. If you only want to move linearly, you also don't need PyTweening at all. How about this solution?

import win32api
import win32con
from time import sleep

Npoints = 5
sleeptime = 0.134 / Npoints

dys = [50, 46, 42, 38, 33, 29, 24, 20, 15, 10, 10]
dxs = [-35, 6, -55, -43, 0, 17, 29, 38, 42, 42, 38]

x, y = win32api.GetCursorPos()

for dx, dy in zip(dxs, dys):
    ddx = dx/Npoints
    ddy = dy/Npoints
    for _ in range(Npoints):
        x += ddx
        y += ddy

        win32api.SetCursorPos(int(x), int(y))
        sleep(sleeptime)

Note that there will still be some very small round-off error and that the cursor will move in a straight line between the points. If the cursor starts at (0, 0), this is the shape it will make (the red crosses are the points where the cursor will be set to):

Shape of the mouse movements

If you wanted to move in smooth curves through the points and you're OK with using numpy and scipy, this will handle that:

import numpy as np
import scipy.interpolate as sci

totalpoints = 50  # you can set this to a larger number to get closer spaced points
x, y = win32api.GetCursorPos()

# work out absolute coordinates of new points
xs = np.cumsum([x, *dxs])
ys = np.cumsum([y, *dys])

# fit spline between the points (s=0 makes the spline hit all the points)
tck, u = sci.splprep([xs, ys], s=0)

# Evaluate the spline and move to those points
for x, y in zip(*sci.splev(np.linspace(0, 1, totalpoints), tck)):
    win32api.SetCursorPos(int(x), int(y))
    sleep(sleeptime)

This results in positions as shown below:

Spline interpolation between points

chthonicdaemon
  • 19,180
  • 2
  • 52
  • 66
1

Question: Tweening, without losing accuracy?

Reference:

  • PyTweening - getLinePoint()

    x, y = getLinePoint(startPoint x, startPoint y, endPoint x, endPoint y, intervall)

    The getLinePoint() function finds a point on the provided line.


  1. Cast your lists, dx anddy, into a list of tuple(x, y)

    dx = [-35, 6, -55, -43, 0, 17, 29, 38, 42, 42, 38]
    dy = [50, 46, 42, 38, 33, 29, 24, 20, 15, 10, 10]
    
    points = list(zip(dx, dy))
    print(points)
    

    Output:

    [(-35, 50), (6, 46), (-55, 42), (-43, 38), (0, 33), (17, 29), (29, 24), (38, 20), (42, 15), (42, 10), (38, 10)]
    
  2. Process this list of points in a double for loop.

    import pytweening
    
    for startPoint in points:
        for endPoint in points:
            x, y = pytweening.getPointOnLine(startPoint[0], startPoint[1],
                                             endPoint[0], endPoint[1], 
                                             0.20)
            x, y = int(x), int(y)
            print('{}, '.format((x, y)), end='')
    
            # win32api.mouse_event(1, x, y, 0, 0)
            # sleep(0.134)
    

    Output: The End Points are allways reached!

    First move from (-35, 50) to (6, 46):
    (-35, 50), (-26, 49), (-39, 48), (-36, 47), (-28, 46), (-24, 45),(-22, 44),
    (-20, 44), (-19, 43), (-19, 42), (-20, 42), (-2, 46), (6, 46)
    
    ... (omitted for brevity)
    
    Last move from (42, 10) to (38, 10):  
    (42, 10), (41, 10), (23, 18), (31, 17), (19, 16), (21, 15), (30, 14),
    (33, 13), (36, 12), (38, 12), (38, 11), (38, 10), (38, 10)
    

Tested with Python: 3.6 - pytweening: 1.0.3

stovfl
  • 14,998
  • 7
  • 24
  • 51