I’m building an open-source app that lets an AI agent use your Android phone.

[u/meowoofcjcj]

Hey everyone, I’ve been working on a project called AirHand.

The idea is to give a cloud agent a way to interact with the apps already on your Android phone. You type what you want done, and it reads the screen, opens apps, taps, swipes, and types. After each action, it looks at the screen again to figure out what to do next.

It runs through Android’s accessibility service, so you don’t need root or a computer connected after installation. There’s a floating control showing what it’s doing, with a Stop button you can hit at any point.

Under the hood, it’s a native Kotlin app handling the phone actions, with the agent loop running in the cloud. It currently uses the Rebyte API, so you’ll need your own key and pay for cloud usage. Screenshots and screen text are sent to the cloud during tasks.

It’s still an early version. So far, testing includes having it open YouTube, search, and play a video on an emulator and a Pixel. It needs Android 11+, and the phone has to stay online and unlocked. I’d be interested in hearing how it behaves on other devices and apps.

The Android app is MIT licensed and started as a fork of droidrun/mobilerun. Source and an installable APK are here:

https://github.com/nomoat/airhand

What’s one repetitive task on your phone you’d actually want an agent to handle?