Human-like mouse movements via Selenium
You can use scipy.interpolate
to interpolate B-spline curves like you can see in this question.
Here I'll use one of the B-spline examples to get values to x
and y
:
import numpy as np
import scipy.interpolate as si
# Curve base:
points = [[0, 0], [0, 2], [2, 3], [4, 0], [6, 3], [8, 2], [8, 0]];
points = np.array(points)
x = points[:,0]
y = points[:,1]
t = range(len(points))
ipl_t = np.linspace(0.0, len(points) - 1, 100)
x_tup = si.splrep(t, x, k=3)
y_tup = si.splrep(t, y, k=3)
x_list = list(x_tup)
xl = x.tolist()
x_list[1] = xl + [0.0, 0.0, 0.0, 0.0]
y_list = list(y_tup)
yl = y.tolist()
y_list[1] = yl + [0.0, 0.0, 0.0, 0.0]
x_i = si.splev(ipl_t, x_list) # x interpolate values
y_i = si.splev(ipl_t, y_list) # y interpolate values
With values of x
and y
, you can move the mouse cursor with ActionChains
:
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
url = "https://codepen.io/falldowngoboone/pen/PwzPYv"
driver = webdriver.Chrome(executable_path="/home/selenium/chromedriver2.25")
driver.get(url)
action = ActionChains(driver);
startElement = driver.find_element_by_id('drawer')
# First, go to your start point or Element:
action.move_to_element(startElement);
action.perform();
for mouse_x, mouse_y in zip(x_i, y_i):
action.move_by_offset(mouse_x,mouse_y);
action.perform();
print(mouse_x, mouse_y)
If you were running this from desktop wand wanted to use an actual mouse movement, with AutoIt you can make mouse movements delayed.
@ODIUM @Guilherme , or anyone still looking for a fix. What ODIUM described in Guilherme's answer as a jump to the first curve position then back to the beginning, then second curve position and back to the beginning again is caused by a small bug in the provided code. It will be fixed by "resetting" the action chain after each perform, like this:
action = ActionChains(driver);
startElement = driver.find_element_by_id('drawer')
# First, go to your start point or Element:
action.move_to_element(startElement);
action.perform();
for mouse_x, mouse_y in zip(x_i, y_i):
# Here you should reset the ActionChain and the 'jump' wont happen:
action = ActionChains(driver)
action.move_by_offset(mouse_x,mouse_y);
action.perform();
print(mouse_x, mouse_y)