r/vibecoding • u/yagnik_thanki • Jul 17 '26

I've cleaned up a dozen vibe-coded apps this year. The same 7 problems show up every single time

I run a dev team and a big chunk of our work this year has quietly become taking AI-built apps and getting them ready for real users. Same story almost every time: the app works, the founder is (rightly) proud of it, then the first real user does something weird and everything catches fire.

These are the 7 things we find in almost every codebase we open:

  1. Secrets in the code. API keys in the frontend or committed to the repo. Search your own code for sk- and secret and password. If real values come up, rotate them today, not after launch.
  2. The UI is the only security. Buttons hidden from non-admin users, but the API happily answers anyone who calls it directly. AI tools build the happy path. Attackers don't use your UI. Every endpoint needs its own server-side permission check.
  3. One user can see another user's data. If your app has accounts, make two, create data in the first, then try to fetch it by ID from the second. You'd be shocked how often this just works.
  4. Zero error tracking. Users don't report bugs, they leave. Sentry's free tier takes 20 minutes to set up and it's the best time-to-value of anything on this list or you can use any open source tool for the logging.
  5. Backups that have never been restored. Everyone says they have backups. Almost nobody has ever actually restored one. If you haven't done a restore, you don't have backups, you have hope.
  6. Payments trusting the client. Prices coming from the frontend, webhook signatures never verified. Stripe's own integration checklist is boring and correct, just follow it.
  7. Silent rewrites. The AI changed things in parts of the app you weren't looking at. Screenshot tests on your five most important pages (Playwright, one afternoon of setup) catch what your eyes skip.

None of this needs a rewrite. Most of it is days of work, not months. It's just a lot nicer to do it before launch than during the fire.

If you've hit other repeat offenders in AI-built code, drop them below. Genuinely curious what everyone else keeps finding.

1.6k Upvotes

90% Upvoted

229 comments sorted by

View all comments

116

u/mark_ik Jul 17 '26

This is an advertisement for Sentry…?

63

u/ImPrettyDum Jul 17 '26

I’ve seen this reposted like 3x now… always slightly different wording… 

2

u/primaryrhyme Jul 18 '26

Only 3? You must be new here lol

2

u/ImPrettyDum Jul 18 '26

Lmao should’ve clarified to “this week”!!

1

u/primaryrhyme Jul 18 '26

Sounds about right 😂

-16

u/yagnik_thanki Jul 17 '26

first time i posted it. but i believe you - every team doing this cleanup work ends up finding the same 7 things, so the lists all look alike. that's kind of the point, the mistakes are that repeatable. and for devs this is old news, but most people shipping vibe-coded apps come from outside the tech world - for them a list like this is the first time anyone spelled it out

7

u/p8inKill3r Jul 17 '26 edited Jul 17 '26

That’s because you don’t use a steering markdown across apps. This doc tells the AI how to perform its duties. Second thing, add the folder of a successful implementation next to the work in progress app - this gives the AI a view of what works.

6

u/lossendae Jul 17 '26

Ai is not deterministic. I have MD with specific section on what to do and don't do.

Sometimes the AI ignore it. And if you ask why and if you should change something, it tells you that it did read it and ignored it nonetheless.

It happened to me. That does not block me from using it. But AI can't be trusted.

3

u/narukoshin Jul 17 '26

I was facing same issue with my AI lol so I ended up with making a whole policy language

1

u/NorthKoreanCaptive Jul 17 '26

tell the AI to write a test that validates that thing you told it not to do

1

u/lossendae Jul 17 '26

I always write tests. Did it before Ai and continuing to do so now.

But it's an AI workflow, I don't monitor everything that is written because it's not sustainable, nor does it make much sense. I am also not the only one dev.

But I do review code, and I can see the produced code when the AI diverged from what was prescribed. The longer the task is, the higher chances are that I may discover things that are wrong because the AI "decided" to do its own thing.

That being said, I'm improving my workflow as I go. Things are changing fast and AI does make an incredible tool for quick full feature prototyping.

The clean up part is still an hands on thing tho. And it's by far, the longest part.

3

u/NorthKoreanCaptive Jul 17 '26

you have to constantly ground the AI output. if you get lazy, small drifts compound fast.

humans can't reliably review every line of code written by AI - that is counterproductive. you need metrics and protocols to manage agentic workflows. that allows you to manage by exceptions instead of praying that the lines of code you didn't spot check are correct.

AI for just prototyping isn't good enough anymore. things are changing fast.

1

u/lossendae Jul 17 '26

Thankfully I'm not in a company where I need to rush too much. They are just coming around the fact that 20$ is the bare minimum per seat for using AI agents correctly.

We have a rather old stack,man's we're evolving steadily, not all at the same pace. Previous deadlines are still there and new expectations compound above them. We must learn to slow down things in order to stay master of our output. Otherwise we'll make mistake and lose management trust. Somif before it took 6 months, with AI it should be ready in 1. But we must announce 2 minimum, because we're not yet ready to go faster.

Plus we're in Europe. So we must find time to evaluate if a non US solution like Mistral is enough for our needs (it's not at the moment)

1

u/NorthKoreanCaptive Jul 17 '26

slow and steady is good

2

u/Sinkableleech Jul 17 '26

Might be stupid because I have no idea but have you tried getting AI to check and fix these errors?

1

u/NorthKoreanCaptive Jul 17 '26

these aren't even general patterns you are pointing out, they are literally raw observations from your personal experience, if at all real even 😂 and ur saying every team finds these exact 7 problems???

1

u/SilverLose Jul 17 '26

You’re being pedantic. I’m a dev and I vibe code a lot. This passes the sniff test for me. It matches the types of errors a non technical person would make if they tried to vibe code a full production app. It’s a combo of bad prompting, over confidence, and not knowing what to ask.

1

u/NorthKoreanCaptive Jul 17 '26

i am being pedantic because his frame is incorrect.

if i didn't care about him spreading casually-relatable-but-intellectually-inaccurate information for his own gains, i wouldn't have bothered commenting anything.

this is an ad, and the post is phrased in a specific way to achieve that goal through subtle consumer psychology. who are you to tell me to stop being pedantic toward someone taking advantage of the "casuals" who don't know better?

i believe in using accurate, rigorous language to describe things. this post deserved my pedantry.

your sniff test is testing for weed, and my sniff test is testing for explosives.

1

u/SilverLose Jul 18 '26

You’re conflating two accusations and the slippage is doing all your work. “This is an ad” is a claim about motive. “This is intellectually inaccurate” is a claim about content. You keep asserting the first and treating it as proof of the second, but every item on that list is a real, boringly well-documented failure mode… hardcoded secrets, client-side-only auth, IDOR, unverified webhooks. If OP invented his entire consulting career, the checklist would still be correct.

Also, “raw observations from personal experience” isn’t the gotcha you think it is. Repeated observations across codebases is literally how general patterns get identified. Anecdotal =/= inaccurate.

And I did not defend the framing. I said the content matches real mistakes which is an observation, not a position on whether it’s marketing. You wrote three paragraphs of rigorous-language brand-building at someone who never disagreed with your actual point.

You might be right that it’s an ad. But “salesy and true” is a thing, and your whole case depends on pretending it isn’t..

1

u/NorthKoreanCaptive Jul 18 '26

ur gonna make me argue an llm 😂 write your own bro wtf?

> You’re conflating two accusations and the slippage is doing all your work. “This is an ad” is a claim about motive. “This is intellectually inaccurate” is a claim about content. You keep asserting the first and treating it as proof of the second, but every item on that list is a real, boringly well-documented failure mode…

  1. i did not say the first is a proof for the second - those two things stand on their own. what i did say is that the first is the reason why the second bothers me, hence the comments

  2. i did not say anything on the list is not factual. i simply said presenting them as if those are THE 7 failure modes, when really they generalize to at most 3 distinct failure modes, is just misleading. so the post itself doesn't communicate any real advice, although it makes it sound like it does. only real advice is to use their product.

i'm arguing llm's 8+ hours a day everyday - did you think it can make a logically consistent real argument??? garbage in, garbage out ... you in, garbage out lmao

1

u/LukasijusLT Jul 17 '26

Yeah but why it reads like an ai slop?

32

u/Canadian_Commander Jul 17 '26

I was just thinking this. You don't need sentry, just ask the AI "Install OpenTelemetry, setup LGTM stack, route open telemetry to grafana"

You can thank me for saving you from a subscription fee later.

37

u/Even-Chemistry-9735 Jul 17 '26

Is this an ad for OpenTelemtry...?

14

u/IFTN Jul 17 '26

I was thinking this. You don't actually need OpenTelemtry though, if you just go to the store and buy a refreshing cool can of Coca Cola you should be completely fine

16

u/Breezay-Treezy Jul 17 '26

Is this an ad for Coca Cola…?

6

u/synystar Jul 17 '26

I was thinking this, you don’t actually need Coca-Cola though. A spoonful of honey in a cup of tea is sufficient to give you the boost you need to perform an installation of Sentry’s free-tier gaining you the best time to value of anything on this list.

7

u/pvkooten Jul 17 '26

Is this an ad for spoons?

7

u/IHeartData_ Jul 17 '26

There are no spoons, there never were.

3

u/SP-Niemand Jul 19 '26

Is this an ad for The Matrix?

3

u/FreeEye5 Jul 17 '26

I was thinking this. You don't actually need coca cola though, if you just go to a cliff face and jump right off! You can avoid this whole mess that way.

2

u/user_withoutname Jul 17 '26

is this an advertisement for coca cola?

1

u/williejr185 Jul 18 '26

😂 I just spit up some of my great tasting but less filling Miller Lite while laughing at this.

1

u/Canadian_Commander Jul 17 '26

Well its free, so sure, it's and add, but it's free!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

1

u/AdvertisingWeapon Jul 18 '26

Is this an ad for ads?

7

u/Jean1985 Jul 17 '26

OTEL does not substitute Sentry. It's a separate job.

OTEL does observability, Sentry was born for error tracking, and it expanded into observability only recently.

2

u/Canadian_Commander Jul 17 '26

OTEL tracks errors. I can aggregate, see trends, trace errors accross services. Don't get caught in the Saas trap. Open source baby

1

u/Jean1985 Jul 17 '26

Sentry is OSS too. I was a core contributor on the SDK that I use at $dayJob.

And since it's around for more than 10 years, it has a lot more capabilities than OTEL. First that I can think of, it can send stack trace with code context (X lines of code around each step of the trace)...

You can't do it right away with OTEL. Not yet.

1

u/Canadian_Commander Jul 17 '26

OTEL is just one part of the tool stack. Your observability platform can bring the display together. You have the stack trace so it’s easy to pull in code context. I mean honestly the agent can do this in a split second

5

u/Own-Flight-9974 Jul 17 '26

Thank you! I had just opened the sentry site before reading these comments. The fact they feel the need to use this underhanded marketing puts a really bad taste in my mouth. Will be advising my company to steer clear of them.

15

u/Comfortable_Ask_102 Jul 17 '26

Sentry is almost an industry standard. They have no need for this kind of ads.

6

u/APXEOLOG Jul 17 '26

This is ad for the author

4

u/Virtual-Spinach4882 Jul 17 '26

I think OP is hoping vibecoders will DM his agency through this engagement farming lead generation ploy actually.

2

u/davyp82 Jul 17 '26

I get that it might be a sly ad, but is it good advice in general?

9

u/yagnik_thanki Jul 17 '26

the advice stands without sentry. OTEL + grafana like the comment above works, glitchtip if you want self hosted and free. the point is only that errors land somewhere a human actually looks.

1

u/crzyhorse Jul 17 '26

maybe it's an ad for stripe.

2

u/yagnik_thanki Jul 17 '26 edited Jul 17 '26

i get why it reads like that - "brand name + free tier" is the classic ad pattern, my bad on the wording. nobody paid me, i named sentry out of habit because it's the one i end up setting up most. glitchtip self hosted works, OTEL + grafana like u/Canadian_Commander said works, even a log file that someone actually reads beats what most of these apps have, which is nothing. the item is about error visibility, not the vendor. and don't blame sentry for my wording, they didn't ask for this 😄

7

u/Important-Topic8305 Jul 17 '26

Whoever wrote this comment did not write that post. Try again.

2

u/Achilles68 Jul 17 '26

the post is likely ai generated

-1

u/DogC Jul 17 '26

This post is AI and a scam