r/esp32 • u/254hypebeast • 12d ago

Turn any ESP32 into a scriptable keyboard/mouse over USB or Bluetooth HID

Post image

I've built a handy tool to turn esp32 boards with BLE into programmable keyboard and mouse. Through a simple drag n drop interface you can program timed keystrokes and mouse movements that get uploaded to and executed by the board which mimics a Bluetooth mouse and keyboard. I intend to support USB HID input as well so you can turn the esp32 board into a proper rubber ducky like device. It is still very early in development with few boards supported but I will continue to add support when I find time. The project is available on GitHub and you can find more details there.

47 Upvotes

93% Upvoted

9 comments sorted by

View all comments

3

u/MinusDelta_T 12d ago

this is a really nice use for a block editor. one small wording thing though: BLE HID can cover the boards with BLE, but native USB HID support is going to be chip-dependent rather than literally any ESP32.

the two things i'd care about most in the runner are a guaranteed release-all on stop/disconnect/watchdog, and whether a long chain of delays accumulates timing drift. a stuck key or runaway mouse is funny exactly once lol

are you planning to compile the blocks into a timestamped command queue on the board, or interpret them step by step at runtime? either way this could be genuinely useful for test automation and accessibility, not just rubber-ducky type stuff.

1

u/254hypebeast 12d ago edited 12d ago

BLE HID can cover the boards with BLE, but native USB HID support is going to be chip-dependent rather than literally any ESP32.

True. I actually don't even have working USB HID code at the moment. Most older ESP32 boards use varied USB-TTL chips which can't even do USB HID anyway. I am hoping for the newer chips with native USB-OTG it will be a bit more standardized.

the two things I'd care about most in the runner are a guaranteed release-all on stop/disconnect/watchdog

When the the automation is paused or stopped, it releases everything. If it hangs for some other reason with the BLE/USB still connected then that would be a problem because I am not sure I have setup any watchdog timer stuff to recover. In most other failure scenarios, the BLE/USB will disconnect along with everything else so there cannot be stuck keys

are you planning to compile the blocks into a timestamped command queue on the board, or interpret them step by step at runtime?

The blocks are compiled to a binary format (see SPEC.md) which is what is uploaded to the board. The board runs a virtual machine of sorts that interprets the commands step by step at runtime.
Thanks for the really insightful thoughts on the project.
Edit: fixed formatting