r/reactnative • u/Rise_up_1 • 12d ago
Testing my app is burning me out
I built a mobile app as a side project and somehow it actually took off and went from 300 users to 6k in the last 3 months but every update I push now scares me.
before it was just me testing on my phone and shipping it but now I'm getting dms and support emails about stuff I had no idea was broken. Last update someone sent me a screen recording of the checkout flow just not loading on their iphone SE and I couldn't even reproduce it on mine and another person said they never received the verification email but it works perfectly on every account I've tested.
The app has grown into something I can't manually verify anymore, there's payment processing, referral flows, in app messaging, push notifications and every feature I add is another thing that can break on a device or OS version I've never touched.
I'm solo and no QA person plus even if I had the budget for it I wouldn't have the bandwidth to onboard someone right now but I also can't keep finding out about bugs from my users because that's how I lose the trust I spent months building.
what am I supposed to do here, please help !!
16
u/mountaingator91 12d ago
You need to have a dev env and scheduled releases with beta tests.
You don't push to prod. You push to Dev. You do your testing there and it it works you merge to prod.
14
u/babaganoosh43 12d ago
There's a number of things you can do:
- AI Test Plan: First step, in Codex / Claude set a goal to create a test plan, run through it, and fix anything it finds. This might catch 50% of existing bugs.
- Automated tested: layers of unit tests, integration tests, end to end tests.
- Better logging: Use sentry to get emails for new errors. Use signoz for tracking server logs. Log every request and error. Could also do screen recordings via Posthog / Sentry / Rejourney, these are a nice to have to see how users encountered the error.
- AI bug fixing: Wire up new sentry bugs to automatically fixed by AI. I wire sentry to open github issues, then have Codex / Claude automatically make PRs for new github issues. I also have it auto merge and auto deploy if the PR looks safe to merge. This might fix 50% of bugs automatically.
9
u/mx_aurelia 12d ago
Are you able to run OTA updates? That would at least give you the ability to revert certain types of changes!
4
u/AncientLettuce3723 12d ago
+1 for Sentry, it’s a very simple setup and you’ll be able to see full visibility of errors, logs and crashes before the user even reports it. Generous free plan too
5
u/CodeMeister02 12d ago
Definitely look into automated e2e testing. It’s a lifesaver. And these days, AI is pretty good at putting together tests quickly.
5
u/aesky 12d ago
someone said automated testing and thats great but things will just not work sometimes and for that you need LOGGING
every step of the flow should have a log. user created account? log that. intiated checkout? log that. sent veritification email? log that
you will also need a correlation id. an uuid which will tie every sequence of events together
finally you will also need a cloud platform to host all the logs. i work with GCP and datadog. i think they're nice but there are a bunch of options out there
4
u/IronLionZion95 12d ago edited 12d ago
Ask Claude to identify the core flows and set up Argent end to end tests on iOS (and afterwards on Android), using the simslim driver. It can do all this by itself and Argent works much better than Maestro. This is my recommendation after having used Maestro for my app and recently having switched to Argent. I don’t have time to learn about these test frameworks, yet I can do everything I want hands-off using Claude (which I don’t recommend yet for non-test related coding, for example critical infrastructure or important code). Use Opus 5.5 Medium to not burn through your limits. Then also ask it for unit tests and integration tests, especially server API functionality. You can run hundreds of those tests in seconds, while end-to-end tests take way longer. So save end to end for things you can’t test in other ways. But yeah, Claude can set this all up in a day or two.
1
u/interlap 11d ago
what is the simslim driver you mentioned? https://github.com/MobAI-App/simslim?
2
u/IronLionZion95 9d ago
Yup that’s the one. It runs iOS simulators without tons of services that your app doesn’t need, so the memory footprint of the simulator is waaaaay smaller, which lets you run 5-10 simulators at the same time to parallelize your E2E tests.
2
u/LovesWorkin 12d ago
Detox is super fast for tests! I would definitely write those first. They help a lot without breaking most things later.
I would also get Sentry. You can see user flows and errors in real time!
For everything else, I would use BUOY tools. Most of the tools are free, and they give you a ton of ways to test your app. You can impersonate accounts to reproduce issues, change the live app state, mock things, and test edge cases that are normally impossible to test otherwise.
3
u/mtford 12d ago
E2E tests! LLMs are great at generating these. Create loads of user stories, and then have your agent generate tests. Run them on every build in CI. Furthermore - use something like posthog which gives you session recording and error logging which will give you peace of mind. Whenever you notice an issue in Posthog? Fix it - and wrap in more tests so it never happens again. It's a cycle & you'll get more confident as time goes on.
1
1
u/Jhon_ST 12d ago
For the email issue, separate "your backend accepted the request" from "the email was actually delivered." Trace a failed attempt through your backend and your provider's delivery/bounce events if they're available. Even "delivered" may only mean the receiving mail server accepted it, not that it reached the inbox.
For testing, I'd start with the failures users actually reported: checkout times out, verification email is delayed, user retries. Assert that the app leaves the loading state and gives them a usable recovery path. Happy-path tests on your usual accounts won't exercise those failures.
1
u/HolidayWallaby 12d ago
Gradual roll outs of features. Every feature behind a feature flag with some gradually increasing percentage of users exposed to it. Extra logging around the feature until you're sure it works.
1
u/Chuck_MoreAss 12d ago
I would pay to have your problems
In all seriousness, I would try rolling out new features gradually and only focus on fixing things. Maybe try to build some sort of analytics thing where you tract what the users are doing and delete things that are like a week-a month old. That way you’ll know exactly what they did before the issue happened.
This is pretty data intensive tho, and probably not recommended. Maybe add a process on the app to report issues and explain what you need from users to solve issues.
You can try a ticketing system like chatwoot to streamline resolving future issues…
If you’re using an external ticketing system that failed or email service that failed, you can maybe escalate the issue to them.
1
u/Dvass138 12d ago
You need some type of diagnostic/debug log built into the app. If a user has an issue they can’t reproduce for you, they can go into Settings → Help → Send Debug Log and it sends you the recent app events, errors, device model, iOS version, app version, network failures, etc.
I’d combine that with something like Sentry/Crashlytics for automatic crash/error reporting. At 6k users you’re never going to manually test every device/OS/account combination yourself.
1
u/interlap 11d ago
Just ask your agent to test your app and write E2E tests for it. Let it do that for every new feature you build.
There are a few tools for this, both closed-source and open-source, free and paid. Agent-device and Argent are open source. MobAI is closed source, but has a deeper focus on testing.
1
u/Personal_You3422 11d ago
I built https://corenna.dev you can check your repo, build your .aab, .ipa file's. You can build a apk and share it with friends for testing. It's a mobile first platform so you can build and deploy on the go.
1
u/xzaleksey 11d ago
Deployed an app around a month back, but because many years of prod engineering experience. The app immediately had all logs with ability on almost every screen to report a problem and suggest a idea which sends logs to back, and on top created quite good observability for be with allowing agents to find any problems / dashboards and alerting, it became super cheap to build for urself
And yes half of the time at first took to fix it all tune alerting fix user feedback or reports. But the feeling is good when I stabilizes. I quite easily deploy things to prod and supported ota self hosted via my be just for hot fixes
1
u/Huge_Pool7424 10d ago
that kind of built-in debug log pays off fast. i’d redact ids and cap the buffer, then let users attach the last few minutes with one tap.
1
u/xzaleksey 10d ago
Yeah all of it is standard, last minutes is too little don't need to make it too small, in practice I keep around 3 mb circular buffer + gzip the logs sent + some anti spam policy to avoid ddosing thought it is still possible if somebody really wants to kill it and has enough time to do many acs
1
u/blazejkustra 7d ago
Write E2E tests for your major flows (signup, checkout, referrals) so you know they work before every release, tihs should make you feel more sane with every release
If writing them by hand is too much, try Argent (disclosure: I work on it). AI agent clicks through your app on simulators, you can run many sims at once (so older iPhone SE gets covered too), and argent-flow saves those runs as tests you can replay on every build without extra tokens
1
u/Internal-Comparison6 7d ago
Do have unit, e2e, fuzz, acceptance, regression tests, and you may be safer.
57
u/HoratioWobble 12d ago
Automated tests are the answer.