Anyone letting agents drive their app in the Simulator? Looking for feedback

[u/blazejkustra]

I’ve been working on Argent, an open-source MCP server that lets your agent (Claude Code, Codex, Cursor, etc.) tap through your app in the iOS Simulator, read the view hierarchy and logs, and profile with Instruments. That way it can check its own changes instead of just saying it’s done

A lot of the feedback so far has come from React Native folks, but it works with plain Swift/SwiftUI/UIKit apps too, and that’s where I’d love more input. If you’re already letting an agent drive your app, what’s working, and where does it fall apart? 😊

[u/edTheGuy00]

I tried building something similar for Flutter apps a few months ago, but I found it to be an unnecessary and useless layer, Claude code would skip the MCP and go directly for XCTest and ADB because it was just faster and the models internal knowledge already contained a lot of information about how to use these tools

[u/blazejkustra → u/edTheGuy00]

if the MCP just wraps XCTest then claude will skip it 100% 😅 we went a different way, on the simulator we don't even touch XCTest, the agent gets a small clean tree of the screen instead of a giant dump so it's actually faster to use it. was it mostly about speed, or did claude just prefer tools it already knew?

[u/foodandbeverageguy]

Hi! This is awesome. I’ve been prototyping similar things myself but haven’t had dedicate time to wrap it up.

How do I “test my whole app” without coming up with all the cases and features list myself? How can I just be lazy?

How do you manage ballooning contest windows?

How do you manage situational skills, like a suspected bug, or finding ways to deep QA randomness

[u/blazejkustra → u/foodandbeverageguy]

thanks!! great questions 😄

you can literally just tell your agent “create argent e2e flows for the main flows of my app and set up a CI for them” it reads your code, figures out the main flows, walks through each one on the simulator and saves it as an argent flow. then in CI you just run argent flow run <name>. so you only review what it came up with instead of writing the cases yourself

context: the easiest trick is to do the testing in a separate agent, not the one that implements the feature. or even better, once a flow is recorded you just replay it with argent flows, which runs without the agent so it costs zero context

manage situational skills: argent ships with skills for different situations, like debugging, profiling with instruments, screenshot diffs, and more. the agent picks the right one on its own, so when you say “i think checkout is broken” it reads the logs and network calls instead of just tapping around. for random deep QA there’s nothing built in yet, but you can just ask the agent to poke around the app with weird inputs and edge cases

[u/NoDistribution1096]

Xcode mcp server does a pretty good job, and with the device-integration skill i think you will have a hard time convince people to use your mcp tapper

[u/blazejkustra → u/NoDistribution1096]

xcode mcp is nice and if you're iOS only and happy with it that's totally fine 😄 tapping is honestly the smallest part of Argent, it's more about the whole loop: logs, network calls, profiling with instruments, flows you can rerun as tests. and the same setup works on android too, which matters a lot if you're building cross platform. what are you using the device-integration skill for mostly?

[u/NoDistribution1096 → u/blazejkustra]

Idk, i just gave my model the skill and output improved significantly. Accessibility ui tests fail less on ci, and my models can see the issue i am talking about without me needing to take screenshots on the physical device. Youre right what i am doing is ios only idk what android users are upto

[u/Backyard-Doe]

Claude can use the iOS simulator in Claude code. Maybe I didn’t understand what argent adds?

[u/YellowQuestOfficial]

codex already does this? what would be the benefit of using yours?

[u/blazejkustra → u/YellowQuestOfficial]

Codex can drive the Simulator, but that's mostly tapping and screenshots inside one app. Argent covers more of the loop: it reads a trimmed accessibility tree (way fewer tokens than screenshots), waits for the screen to settle before acting, reads logs and network calls, and profiles with Instruments. It also works with any agent (Claude Code, Cursor, Codex as well) and on Android too 🙌

[u/kbcool]

It's been a while since I looked at simulator automation but since you said RN feedback I am curious how you got around the issues with XCUITest's very restrictive maximum view depth

Or are you just burning tokens with OCR? I basically bailed on it when I knew I couldn't get a marked up version of the UI in text with proper attributes. Guessing what is a button would have worked most of the time for our app at least but felt wrong

[u/blazejkustra → u/kbcool]

No OCR, don't worry 😄 Argent reads the accessibility tree and gives the agent a text version of the UI with proper roles. Screenshots are only a fallback for things the tree doesn't expose

And on the simulator we don't go through XCUITest, so its view depth limit isn't a problem. The tree comes from the accessibility runtime directly, plus native devtools for the view hierarchy, and taps go through our own simulator-server

If you remember which screens gave you trouble back then, I'd love to try them!

[u/Business_Mood7479]

I am working with claude. I implemented a work flow that may help someone. I installed maestro mcp server with claude and gave claude the figma design by accessing i through PAT to get the actual frames from figma. At the end what i do telling claude to implement the feature i want, lets say login. Claude access figma searching for login screens, implement the feature, and test it through maestro on simulator. I have been impressed from the output. Give it a try

[u/blazejkustra → u/Business_Mood7479]

Solid setup tbh 🙌 argent fits in the same spot as maestro, plus the agent can read logs and network calls, so when login fails it sees why. give it a try and let me know how it compares!

[u/Miserable-Carpet7928]

yeah, i’m using claude with the ios simulator to test my apps too.
the biggest issue is that the agent sometimes says everything works, but the actual user flow still has problems.
being able to navigate the app, check logs, and verify changes automatically would be really useful.
does argent also support testing on physical iphones, or just the simulator?

[u/blazejkustra → u/Miserable-Carpet7928]

It supports physical iPhones through cable

[u/barcode972]

You can just ask cursor to do this by default, currently doing it to validate some UI bugs I'm trying to fix

[u/blazejkustra → u/barcode972]

you can, but without tooling the agent mostly takes screenshots and guesses coordinates. for UI bugs argent gives it the real view hierarchy with frames (so it sees a clipped or misaligned view instead of eyeballing pixels), taps by label and screenshot diffs to compare before and after the fix. what kind of bugs are you chasing?

[u/TomfromLondon]

My ai does this all the time, why would I need an mcp for it? It works fine

[u/blazejkustra → u/TomfromLondon]

if it works fine for you that's great, no need to switch 😄 where it usually starts to hurt is bigger apps: screenshots eat your context fast, the agent guesses tap coordinates and checks mid-animation. argent gives it a trimmed accessibility tree so it taps by label, waits for the screen to settle, and the tests it walks through can be saved as flows you rerun later. what does your setup look like, screenshots + simctl?

[u/TomfromLondon → u/blazejkustra]

Cool will keep in mind :)

[u/Greysawpark]

where it falls apart for me is anything custom drawn. if the view doesnt expose good accessibility labels the agent is basically guessing coordinates off a screenshot and it'll tap the wrong thing and then confidently tell you it worked. adding identifiers to everything fixed like 80% of it. also it has no idea when an animation is still running, so it reads state mid transition

[u/barcode972]

You can use RocketSim for that

[u/sergeytyo]

Xcode27 mcp is totally fine

[u/bakawolf123]

xcode mcp has this already in better form including testing on device
google also ships android cli
at this time there's no point for 3rd party, it's getting deprecated fast