Equivalent to xdotool for Wayland

Question

I'm looking for a solution working in Wayland to get mouse position, move it and click inside a bash script, all things xdotool can do for X server.

xdotool getmouselocation

is still working, but

xdotool mousemove 
xdotool click

have no impact on the mouse cursor in GNOME Wayland session.

See also https://superuser.com/questions/1032270/wayland-alternative-for-xorgs-xdotool and https://unix.stackexchange.com/questions/381831/keyboard-emulation-in-wayland — Ben Creasy, Feb 18 '18 at 09:32
If anyone has functional Python tools to do this stuff too, that would be useful and those should be mentioned here too. So far I'm really struggling though: Stack Overflow: pynput library not working as expected in Python to press Windows + D key — Gabriel Staples, Jun 04 '23 at 08:19

score 9 · Accepted Answer · edited Sep 07 '22 at 19:05

9

There is the ydotool package for Wayland:

Package ydotool

Generic Linux command-line automation tool (no X!)

https://github.com/ReimuNotMoe/ydotool

Performs some of the functions of xdotool(1) without requiring X11 - however, it generally requires root permission (to open /dev/uinput)

Currently implemented command(s):

type - Type a string
key - Press keys
mousemove - Move mouse pointer to absolute position
click - Click on mouse buttons

N.B. optionally, you can enable and start the ydotoold daemon with:

systemctl enable ydotool
systemctl start ydotool

edited Sep 07 '22 at 19:05

Pablo Bianchi

15,657

answered Dec 10 '20 at 12:00

WinEunuuchs2Unix

102,282

Great answer. Important to note is that ydotool is not just for Wayland. From the official github repo: "ydotool is not limited to Wayland. You can use it on anything as long as it accepts keyboard/mouse/whatever input. For example, X11, text console, "RetroArch OS", fbdev apps (fbterm/mplayer/SDL1/LittleVGL/Qt Embedded), etc." I've tested it on Ubuntu 22.04 with the X11 window manager, for instance, to press the Super + D keys to show the desktop, and it works great. – Gabriel Staples Jun 05 '23 at 01:57
I wrote a tutorial here (How can I write a program to press keys, such as Windows + D, in Wayland? (replace xdotool in Wayland)) and on my website here (Tutorial: Getting started with ydotool to automate key presses (or mouse movements) in Linux) for anyone wanting to get started with ydotool. It was tricky and took me a long time to write this tutorial, so hopefully this helps a lot of people. – Gabriel Staples Jun 24 '23 at 01:31

score 9 · Answer 2 · edited Nov 18 '17 at 20:10

9

This is because such features have been explicitly removed from Wayland for security reasons. The major concerns were reading other programs input and allowing fake input to be sent to other programs which would allow different attack vectors.

Some window-managers might implement some sort of macro feature in the future but as of now there is no such feature that I know of.

Implementing this would mean to implement it for each of the different window managers which surely will take still a while.

I suggest using Xorg instead of Wayland for now if you need to use such features.

edited Nov 18 '17 at 20:10

dessert

39,982

answered Nov 18 '17 at 18:02

Videonauth

33,355
17
105
120

3

I don't know if you are aware of any of the discussions, but there was a RFC on a mailing list: RFC: Interface for injection of input events and Add an API for taking screenshots and recording screencasts. It sounds like the challenges could be addressed with some additional hardening work. – Ben Creasy Feb 18 '18 at 09:36
1

Is this still up to date? If no click/type automation is possible, it would be an absolute blocker against Wayland for me. (Also, it's never great when someone tells the user what they're allowed to do on their own computer. This is not Windows.) – Fabian Röling Jul 27 '21 at 09:43
1

So, re ydotool, for security reasons, automation on wayland now requires a daemon with root privileges that affects all users, instead of the fake input tool being contained to a specific x11-session of a specific user. The issue for taking screenshots is still open 5 years later: https://gitlab.freedesktop.org/wayland/wayland/-/issues/32 – Arne Babenhauserheide Apr 19 '22 at 11:12

score 2 · Answer 3 · edited Sep 07 '22 at 21:26

evemu from the evemu-tools package can emulate devices like mouse, touchpads and keyboards in wayland. It cannot "read" what is happening on the screen but can easily move & click and "blindly" interact with the session.

sudo evemu-describe # list devices

if your mouse is /dev/input/event5 from the output of above command the following will move mouse 50 pixels xy then right-press & let go of button:

sudo evemu-event /dev/input/event5 --type EV_REL --code REL_X --value 50
sudo evemu-event /dev/input/event5 --type EV_REL --code REL_Y --value 50 
sudo evemu-event /dev/input/event5 --type EV_KEY --code BTN_RIGHT --value 1 
sudo evemu-event /dev/input/event5 --type EV_KEY --code BTN_RIGHT --value 0

You can also record a particular sequence and re-play it:

# press ctrl-c to stop recording
sudo evemu-record /dev/input/event5 /tmp/mouse-sequence
to replay session
sudo evemu-play /tmp/mouse-sequence

Pablo Bianchi · Answer 4 · 2022-05-03T04:47:05.953

2

There is the wtype package for Wayland. It's available on apt repos on Ubuntu 22.04.

Seems a simpler approach (no need for ydotoold/systemd service), but for the time being, mutter (GNOME compositor) doesn't work with it.

Check also Evemu, a freedesktop project that records and replays device descriptions and events.

edited May 03 '22 at 04:47

answered Apr 26 '22 at 18:53

Pablo Bianchi

15,657

Equivalent to xdotool for Wayland

4 Answers4

Package ydotool

to replay session

Linked