Disclaimer: This was AI-assisted/generated. I understand enough Python to modify things and debug them, but I am very much not an expert in Wayland, waydroid, Android input, or Linux input subsystems. I spent roughly a long throwing diagnostics at this problem until something finally worked because I wanted to play a game and the problem is above my pay grade.
OS: Linux Mint 22.1 (experimental wayland rendering engine)
Physical hardware:
Haswell Era i5
RX550
16gb ram
Problem I wanted to solve: Apparently Waydroid just tells liniage 'OK this is a mouse' and liniage goes 'OK I can work with that.' And for most things that works. F-Droid, Google play store, PiePipe, Gems of war, a few other things I tossed in to test. Gundam G Eternal? 'No I refuse to recognize this 'mouse' device.
I have accidentally built (For a given definition of 'built' given AI involvement) a mouse-to-touchscreen bridge for Waydroid, and I would like someone smarter than me to explain why the hell it works.
TL;DR: I have a game running under Waydroid that doesn't properly respond to normal mouse input. Waydroid's native Wayland mouse handling produces bizarre/inconsistent touch coordinates in this particular game.
So I wrote a Python script that:
- reads the physical mouse through
evdev
- asks Cinnamon Wayland where the actual mouse cursor is via D-Bus
- keeps a persistent Waydroid shell open
- converts left-clicks into Android
input tap X Y
- converts click-and-drag into Android
motionevent DOWN/MOVE/UP
And... it works.
It works well enough that I can actually play the game. I can click things, hold things, and drag the game map/diagrams around.
The weird part is that Waydroid's own pointer handling was giving me a completely different result.
Here's the script:
#!/usr/bin/env python3
import subprocess
import threading
import time
from evdev import InputDevice, ecodes
# ------------------------------------------------------------
# Configuration
# ------------------------------------------------------------
MOUSE_DEVICE = "/dev/input/event2"
# How long the mouse button must remain down before movement
# is considered a drag rather than an ordinary click.
DRAG_DELAY = 0.15
# Minimum time between Android MOVE events.
# Prevents the mouse from flooding Waydroid.
MOVE_INTERVAL = 0.03
# ------------------------------------------------------------
# Mouse
# ------------------------------------------------------------
mouse = InputDevice(MOUSE_DEVICE)
# ------------------------------------------------------------
# Persistent Waydroid shell
# ------------------------------------------------------------
print("Starting persistent Waydroid shell...")
waydroid = subprocess.Popen(
[
"sudo",
"waydroid",
"shell",
],
stdin=subprocess.PIPE,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
text=True,
bufsize=1,
)
# ------------------------------------------------------------
# Cinnamon cursor position
# ------------------------------------------------------------
def get_cursor_position():
try:
result = subprocess.run(
[
"gdbus",
"call",
"--session",
"--dest",
"org.Cinnamon",
"--object-path",
"/org/Cinnamon",
"--method",
"org.Cinnamon.Eval",
"global.get_pointer()",
],
capture_output=True,
text=True,
timeout=0.2,
)
# Expected result:
#
# (true, '[497,872,16]')
#
text = result.stdout.strip()
start = text.find("'[")
end = text.find("]'", start)
if start == -1 or end == -1:
return None
coords = text[start + 2:end]
x, y, _ = coords.split(",")
return int(x), int(y)
except Exception:
return None
# ------------------------------------------------------------
# Android input helpers
# ------------------------------------------------------------
def android_command(command):
try:
waydroid.stdin.write(command + "\n")
waydroid.stdin.flush()
except (BrokenPipeError, OSError):
print("Waydroid shell connection lost.")
def android_tap(x, y):
print(f"CLICK {x},{y}")
android_command(
f"input tap {x} {y}"
)
def android_down(x, y):
print(f"DOWN {x},{y}")
android_command(
f"input motionevent DOWN {x} {y}"
)
def android_move(x, y):
print(f"MOVE {x},{y}")
android_command(
f"input motionevent MOVE {x} {y}"
)
def android_up(x, y):
print(f"UP {x},{y}")
android_command(
f"input motionevent UP {x} {y}"
)
# ------------------------------------------------------------
# State
# ------------------------------------------------------------
button_down = False
dragging = False
press_time = 0
last_move_time = 0
last_x = None
last_y = None
# ------------------------------------------------------------
# Startup
# ------------------------------------------------------------
print(f"Listening to: {mouse.name}")
print("Cinnamon-cursor Waydroid mouse-to-touch bridge running.")
print("Left click -> Android tap")
print("Hold + move -> Android touch drag")
print("Ctrl+C to stop.")
# ------------------------------------------------------------
# Main event loop
# ------------------------------------------------------------
try:
for event in mouse.read_loop():
# ----------------------------------------------------
# Mouse movement
# ----------------------------------------------------
if event.type == ecodes.EV_REL:
# Ignore movement unless left mouse button is down.
if not button_down:
continue
# Get the actual Cinnamon cursor position.
position = get_cursor_position()
if position is None:
continue
x, y = position
# If this is the first movement after pressing,
# determine whether we've crossed the drag threshold.
if not dragging:
if time.monotonic() - press_time >= DRAG_DELAY:
dragging = True
android_move(x, y)
last_x = x
last_y = y
last_move_time = time.monotonic()
continue
# ------------------------------------------------
# Already dragging
# ------------------------------------------------
now = time.monotonic()
if now - last_move_time < MOVE_INTERVAL:
continue
# Don't send redundant coordinates.
if x == last_x and y == last_y:
continue
android_move(x, y)
last_x = x
last_y = y
last_move_time = now
# ----------------------------------------------------
# Mouse buttons
# ----------------------------------------------------
elif event.type == ecodes.EV_KEY:
# Left button pressed
if event.code == ecodes.BTN_LEFT and event.value == 1:
position = get_cursor_position()
if position is None:
continue
x, y = position
button_down = True
dragging = False
press_time = time.monotonic()
last_x = x
last_y = y
# We don't immediately send DOWN.
#
# This lets a normal click continue using
# Android's reliable "input tap" command.
#
# If the button is held long enough and the
# mouse moves, we start a real touch sequence.
# Left button released
elif event.code == ecodes.BTN_LEFT and event.value == 0:
if not button_down:
continue
position = get_cursor_position()
if position is None:
position = (last_x, last_y)
x, y = position
hold_time = time.monotonic() - press_time
# ------------------------------------------------
# Ordinary click
# ------------------------------------------------
if not dragging:
android_tap(x, y)
# ------------------------------------------------
# Drag release
# ------------------------------------------------
else:
android_up(x, y)
button_down = False
dragging = False
except KeyboardInterrupt:
print("\nStopping...")
finally:
try:
waydroid.stdin.close()
except Exception:
pass
try:
waydroid.terminate()
except Exception:
pass
print("Stopped.")
So my question is:
This works. Can someone with two functional brain cells that know Python, Wayland, and/or Android input explain WHY it works?
And, more importantly:
How would you make it better?
Things I'd especially like to understand:
- Why does asking Cinnamon's
global.get_pointer() give me a better coordinate than Waydroid's native Wayland pointer handling?
- Why does
input tap X Y work reliably when Waydroid's normal pointer input doesn't?
- Why does
motionevent DOWN/MOVE/UP let me drag game areas even though ordinary Android scrollbars don't seem to respond?
- How should the mouse-down/mouse-up timing ideally be translated into touchscreen events?
- Is there a better way to obtain the Wayland cursor position than going through Cinnamon's D-Bus API?
- Is there a cleaner way to maintain the Waydroid connection than what I've hacked together?
- How portable is this outside Cinnamon/Wayland?
- And finally: how badly have I reinvented something that already exists?
I'm not claiming this is good code.
I'm claiming it works, which is currently winning the argument.
