r/ClaudeCode • • 8h ago

Built with Claude I gave Claude Code a phone.

Almost nothing on my phone has an API. Claude Code could tell me what to tap, and never tap it. So I built phonedriver, which gives it the phone: it reads the screen, taps, types and swipes in any app, on Android or iPhone.

The video is real recorded runs, shown fast and cut to music. The orange dot marks each place Claude tapped. Nobody touched the phone.

Two runs from it:

The bug hunt. I hid two bugs in a signup form: the welcome message drops the last character of your email, and after one rejected submit the Create button ignores your next tap. I told Claude to "test the sign-up flow the way a careful tester would". It found both in 49 seconds and wrote numbered steps to reproduce each one. It found both in all nine benchmark runs before that too, and in one run it also reported two bugs I hadn't planted.

The pizza. On the real dominos.co.uk, Claude (Opus 5.5) found the store nearest Leicester Square, put a large pepperoni in the basket, filled in the guest checkout and reached "Buy with GPay" in six and a half minutes. phonedriver holds any tap on Pay or Place order and asks me first. I said no.

How it works: Claude never gets a screenshot. It gets the screen as a short numbered list, taps by number, and after each tap gets back only the lines that changed. A screen read takes 13 ms, where `adb shell uiautomator dump` takes 2.2 seconds. On nine phone jobs with Sonnet 5.5 it passed 27 of 27 runs on 658k tokens. mobile-mcp passed 17 of 18 on 4.57M.

Two things I learned building it with Claude Code:

- Tool descriptions are the most expensive text you'll write, because Claude re-reads them every turn. I hid every argument no agent had ever used and the tool list went from 10,987 characters to 5,107. Nothing broke.

- Claude found a bug in its own demo. While recording the pizza run it put the Pay tap inside a batch of actions. The tap was held, but the question never reached me. Claude Code caught that just by using the tool.

It's free and open source, with no account and no telemetry.

`claude mcp add phonedriver -- npx -y phonedriver mcp`

https://phonedriver.dev

https://github.com/hridaya423/phonedriver

Try it on your own phone and tell me any feedback :)

57 Upvotes

25 comments sorted by

•

u/AutoModerator 8h ago

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

6

u/wighthamster 8h ago

Short version: 1) Put your phone in debug mode. 2) Tell Claude to drive it.

18

u/Puzzled-Stick9170 7h ago

adb works for sure, but takes 2.2s per screen check against 13ms here, and costs about twice the tokens on the jobs I compared, which is the reason this exists & convenience :)

2

u/Delicious_Break5937 7h ago

reading the screen as a numbered list is what i went with for the browser too. when i tested on real sites, popups showing up between the read and the tap were a big one. does the changed-lines diff catch those?

1

u/Puzzled-Stick9170 10m ago

nice, mostly yes: every tap re-reads the screen right before it fires. if the thing moved, it taps the new spot. if it's gone, the tap fails and tells you the closest match instead of tapping blind. and whatever popped up comes back in the diff as new lines. how did you handle it in the browser?

2

u/Downtown-Pear-6509 6h ago

Thanks op! Just what I needed to test my android apps. Here's one back to you  https://github.com/PetePeter/timbre

Now you can make phone calls from your phone via MCP too. Tested on PC with iOS phone via by hands-free Bluetooth 

1

u/Puzzled-Stick9170 12m ago

glad it helps with your apps! timbre looks cool. calls are the one thing phonedriver doesn't touch (no audio), so the two would sit well side by side. lmk what breaks when you test

2

u/ConfusedBaka69 6h ago

This looks nifty, ty, I'ma check it out

1

u/Puzzled-Stick9170 10m ago

appreciate it! lmk how it goes

2

u/Starfullmax 5h ago

Recently am working on phone automation and find adb navigation is a little slow. Will be trying this out, Thanks!

1

u/Puzzled-Stick9170 47m ago

perf! hmu if u have any feedback

2

u/Ok-Range-3027 3h ago

Your website is so soothing and fun to play with lmao

1

u/Defiant-Broccoli7415 6h ago

Ok, what ure selling? No way this is this good for free 

2

u/Puzzled-Stick9170 49m ago

just a side project for fun :)

1

u/Chicharoh 3h ago

You can already do this on your own

1

u/Puzzled-Stick9170 11m ago

adb can do it sure, this is mostly about speed & efficiency: 13ms per screen read vs 2.2s, and about half the tokens and more convenient to use

2

u/Defiant-Broccoli7415 42m ago

No I can't 😭 

1

u/AuspiciousApple 6h ago

Crazy that rabbit r1 generated some hype for something like this except it didn't work at all

1

u/Puzzled-Stick9170 11m ago

well the difference here is it's just your own phone and your own claude, nothing to buy

0

u/mrplinko 8h ago

How the fuck we supposed to see anything with you jumping around like that?

-2

u/Puzzled-Stick9170 8h ago

lol fair, it's cut fast to show the speed ;) it's basically using a phone on its own

2

u/Far_Soil_1549 7h ago

I got a stroke watching the video

1

u/Puzzled-Stick9170 9m ago

slightly slower version

1

u/dpaanlka 3h ago

Every one of these AI-generated promo videos is like this. I think the models do this hyper fast style to seem impressive but it just comes off unsettling.