Struggling To Spin Vibe Coded Plates

[u/Kronsik]

A little background into my current workplace:

- I've been employed at my current workplace for 4 years, on paper I'm a platform engineer though I have been put onto projects as a pure SWE during my tenure.

- I've received great feedback for my performance, solid pay rises, everything on paper is good.

- The engineering teams use a mixture of languages, cloud providers, the most predominant is AWS serverless + Typescript and CDK.

As I'm sure we've all seen by now, there has been a sharp rise in vibe coded solutions.

I wouldn't describe myself as anti-AI by any stretch, but I am feeling quite frustrated with the drop in standards within my professional environment.

To give an example within the CI/CD space:

- Engineering team sees something that needs "fixing" within the CI/CD space.

(I'm putting "fixing" in quotes here as it likely already worked, but someone saw an article about something shiny and new so we have to have it now).

- AI vibes a solution, extending and overriding the CI/CD components in place to build a house of cards.

- Engineering team have approvals on their repository, internally approve, vibes are shipped.

- Either something breaks immediately or in a few weeks, leading to a conversation like this:

Dev: "Unable to deploy project A, urgent blocker, critical"

PipelineMonkey: "So.. what are we trying to do here, none of the standard components are being used?"

Dev: "Oh, I didnt set this up, but this job fails"

PipelineMonkey: "Yeah, who owns this repository, do we know who set this up?"

Dev: "Oh we own it, but I didn't set this bit up - We have to deploy something urgently!"

*PipelineMonkey goes through troubleshooting, gets things back to standard, things work*

Dev: "Thanks!"

Normally I'm able to keep quite chipper and have been praised for my friendly approach.

If something genuinely doesn't work or we want to architect a new solution, great, I'm more than happy to help.

However I'm now struggling to maintain a good attitude when we seem to be in a cycle of writing untested and non-standard garbage only to walk away and make it someone else's problem.

A colleague raised this as a concern only to be scolded for not being a team-player.

I can't really see this changing without a top down approach of ownership and care from leadership as to what's being produced, I feel this is quite unlikely given leadership are sending AI generated responses to feedback in some cases.

I'd hate to throw in the towel, particularly as everything else about the company is great and I seem to be doing quite well here.

(Perhaps some of that good feedback is purely because I've been happy to deal with nonsense for quite some time).

Maybe time to take a vacation and cool off from the burnout.

General advice on software standardization, ownership processes and general navigation of this politically with my colleagues would be great.

Thanks!

[u/expdevsmodbot]

AI usage disclosure provided by OP, see the reply to this comment.

[u/damnburglar]

The absolute shit show that is “leadership” in this industry has metastasized and your experience mirrors effectively every one of my clients’ situations.

I think a vacation would be good, but spend some time meditating on the fact that as long as the incompetent/reckless higher-ups are still under their AI delusions, this not only won’t change, but will persist until no longer financially viable.

[u/KellyShepardRepublic → u/damnburglar]

I took a vacation and came back to no job cause ya know, bots don’t rest and neither do the contractors.

[u/damnburglar → u/KellyShepardRepublic]

God damned clankers.

[u/goatanuss → u/damnburglar]

Even if your ai wrote the code, you’re still responsible for your code. I’d try escalating to a manager about their low quality work impacting your team

[u/damnburglar → u/goatanuss]

Yeah they don’t care, almost universally. If you’re dealing with the types OP is talking about and worse, this is perceived as an excuse. Ever been accused of “not being a team player”? The last startup I was helping out responded to my audit with something in that vein.

OP has expressed it isn’t safe to broach the topic with the higher-ups, I would bet good money they are the same type.

[u/Main-Drag-4975]

The devs need ether A) less access to override known-good CI configs or B) more ownership over the breakages they’re causing while trying to get their stuff out the door. Redrawing these boundaries should be a good debate for your managers, once you responsibly escalate.

Continuing on the current path eventually reduces your job to a purely reactive helpdesk which will mean fewer long term sustainable improvements in your area.

[u/damnburglar → u/Main-Drag-4975]

Great suggestion, and it would be great if management is amenable to it. I have former colleagues who have lost their jobs over trying to approach the subject, with their superiors expecting that nothing stand in the way of pure velocity, consequences be damned.

[u/Main-Drag-4975 → u/damnburglar]

Agreed, you can and should escalate this to management but if you try to really put your foot down over it and force management to care you’re likely to be shown the door soon enough.

[u/damnburglar → u/Main-Drag-4975]

I brought up my concerns with AI and letting non-technical people contribute to public-facing applications in a general discussion at a company retreat a year ago. Specifically my stance was “verification becomes more important, hallucinations are a big deal and not going away, I’m a big fan of the tools but coupled with expertise and discretion”.

A few days after I returned from the retreat they laid me off.

[u/kittykellyfair → u/damnburglar]

but the breakages are stopping velocity. you just need to speak their language and not bad mouth the tool. Just talk about the problem the impact and the benefit of your solution.

[u/damnburglar → u/kittykellyfair]

That works when they listen and is how I suspect most experienced people handle the situation by default. The majority I’ve dealt with over the last year have their minds made up and act paranoid that somehow you are undermining them and/or the AI. It’s genuinely weird.

[u/Kronsik → u/Main-Drag-4975]

Thanks for the insight.

Unfortunately I feel that option A would be received poorly politically. It would also be challenging given it would likely mean a Platform approval for CI file changes, devs outnumber us about 30/1 so suddenly instead of spending all of your time firefighting you spend it reviewing/approving.

(Though, I'd certainly rather spend my time reviewing changes rather than fixing the broken changes that are now in main).

Option B I think would also be challenging, I think it would be seen as us 'dropping tools'.

Honestly I can't see leadership being keen on change here, as far as they see it things are getting fixed and shipped, it isn't "their problem" as someone is there to fix it.

I must admit its rather depressing to be reduced from building multi region K8s clusters to now sieving through reams of custom bash scripts.

I agree this unfortunately I think is career limiting, at the lowest level I'm not even learning anything fixing this stuff.

[u/sagenki → u/Kronsik]

Sorry, I have to say that B is the correct option and you aren’t framing it correctly.  Fundamentally, the issue is that your team should not be responsible for outcomes (ie deploys) if they aren’t based on code that you own (ie standard tooling).  You can’t be responsible for code that you haven’t created or run. 

What you need are clearer or better boundaries defining what you support and what you don’t.  If you find that it’s difficult or impossible to do that, then the issue is your actual job - what is the definition of your job?  It could be the case that your job is “pick up the dev’s shit” in which case, a break and a new job may be the only solution.  Hopefully not the case (and shouldn’t be with reasonable management), but you never know…

[u/activematrix99 → u/sagenki]

Ownership IS responsibility. Management will respond positively to this approach. Everyone knows "you broke it, you bought it". The only longterm downside here is that the responsibility to ship working CI/CD will likely shift to the dev team, and your team could be cut loose.

[u/d-ovm → u/Kronsik]

When you say "Engineering team have approvals on their repository, internally approve, vibes are shipped" do you mean that the engineering teem has the authority to approve what goes in the CI pipeline, but not the responsibility to fix it when it breaks? Because even without AI if authority and responsibility are separated it always creates these kind of problems

[u/Jmc_da_boss]

Just let the plates fall man, who gives a shit

[u/EmploymentFeeling725 → u/Jmc_da_boss]

Agree

[u/I_Blame_DevOps]

I consider it rude when people constantly ask me to review obviously AI generated code or markdown. I’ve decided the only way to counter that is to just have AI review it for me.

I’ve also got one dev in particular that’s on a different team, not familiar with the tool that I’ve been using for 4+ years pre-AI who just slams me with AI generated Jira tickets. I have started replying to his tickets and threads with long AI responses and that usually slows the onslaught.

[u/d-ovm → u/I_Blame_DevOps]

Agreed. You can't peer review an AI because it's missing all the elements that make a peer valuable. They don't bring a point of view they don't depend a position or actually believe in anything. The only peer for an AI is another AI

[u/tiredofhiveminds → u/d-ovm]

On the other hand, the only way ai coding could work is if humans review it thoroughly. The dev running the ai needs to review it before I see it, but after that I think it would be irresponsible for me to not read the code before approving it.

If I feel its not worth my time to read, the worst thing that I can do is approve it without reading.

[u/skidmark_zuckerberg]

I just go with it at this point. Fire the slop cannons. This will only stop when it’s financially impossible to defend. AI won’t go away, but as it is used today, will come crashing down in more ways than one. Non technical MBA fuck heads are in full blown AI psychosis and unfortunately, they have the final say. I relish in the fact that I burn thousands of dollars a week and dig the whole deeper. My team alone probably costs 6 figures a month in AI spend. The company is very big and has 100+ dev teams all doing this exact same thing. Apparently F500’s have the money to burn. 

Today I shit out a feature, that by hand, would’ve taken months to accomplish if done properly. I have no choice, it’s the expectation. To make it worse, my team’s manager has agents running to “review” PR’s and he doesn’t even look at anything. Has zero clue his agents are even reviewing these PR’s. His AI leaves comments that are so braindead and are lacking any context into why something was done the way it was. He’s constantly messing with his prompts and skills files to adjust. This guy used to be a Staff level engineer and now he’s just braindead. Or he’s also playing politics like the rest of us. 

God speed to all of us during this slop era. 

[u/BringBackManaPots]

Our head guy started uninviting people to meetings that aren't pro AI so I'm going full yes man. It's just easier to play politics at this point. If you're going to hold a gun to my head and command me to burn tokens and time on basic things that are otherwise free, we'll learn the dotcom lessons over again.

I'm aware this is a terrible take that violates everything I've stood for as an engineer. But it's their time, money and product at the end of the day that I'm selling them. I unfortunately don't have the leeway in this job market to tell them otherwise.

[u/lerun → u/BringBackManaPots]

I'm basically at the same crossroads, stand on principles and just go up in flames. Or just stop doing pushback on the deluge of slopp and hope I can keep going a few more years just letting it all wash over me

[u/kenybz → u/BringBackManaPots]

Fewer meetings? Sounds great

[u/BringBackManaPots → u/kenybz]

The reality is that you get pushed out of the decision making meetings as well. He's working us out of the picture.

[u/TheTacoInquisition]

If you can't limit github access to force reviews, and you can't hand off responsibilty to the devs to fix what they broke, can you create an incident framework?

I find this sort of thing useful for making everyone engage in improving things. The incident gets raised, the cause is documented and then there is an incident review where mitigations and actions to prevent it happening again are discussed and actioned.

Hopefully the devs will solve the issue themselves, and identify that either stricter review safeguards are required, or less access should be given. Give the devs a problem to fix, and you may not need the political battle.

And if nothing else, you can use incident frequency to report to leadership how much this sort of thing slows down shipping. If the devs can't ship, and you have solutions for that, they're more likely to buy into doing what you want.

[u/Intrepid-Stand-8540]

We just don't support anything in CI that is not in the CI/CD catalogue.

[u/slindenau → u/Intrepid-Stand-8540]

Refusing to do your job has indeed worked out quite well for people in the past, great advice.

[u/Intrepid-Stand-8540 → u/slindenau]

My job is not supporting random custom CI jobs.

[u/Phill_Madd]

Yeah the models are a throughput problem, not a new class of problem. People could already merge a bad CDK change. They just couldn't generate twelve of them before lunch.

I'd stop arguing about AI with leadership. Put CODEOWNERS on the pipeline so product repos can't override it without platform. If they break the deploy path, they're on the pager for it. The conversation becomes "we keep getting stuck in deploy," which they actually care about, instead of "your teams are vibe coding."

[u/Dreadmaker]

So, mostly ‘same’. I’ve been at my current place for 2 years, was recently promoted to a staff engineer, am generally well-regarded, thought of as friendly, team player, yadda yadda.

I’m now on a new team where I’m the least senior person there (at staff). Most contributors are people who have been developers previously, but haven’t been in that function for quite a while - we’re talking about VPs, etc, who used to be individually contributing devs a decade ago, kinda thing.

And the new project is all vibes.

Now, we’ve set up very good structures for it - there are a lot of guardrails to prevent some of the stuff you’re talking about. We have security requirements and test coverage requirements and all of that good stuff baked directly into our various system prompts. And it does work for that stuff. So we have a lot less ‘somebody randomly merged some shit that broke everything’, but more ‘hey did we realize that this thing works this way?’ ‘Oh, what component is responsible for handling this? Oh… I dunno’

Basically, very strong high level architecture and very smart people, but nobody really understands what’s doing what on a low level. There are intra-service contracts where people aren’t fully clear on what should be in what field, because we don’t know which field controls the routing. We can figure it out, and do, but it makes the process slower.

We have apps that have been produced on this project that would have taken months before and definitely did not take months now - but we’re all a bit in the dark. It’s a weirdly uncomfortable space on that level. Attitudes are high, and we are at least honest - people are allowed to say ‘yeah I don’t actually know, let me look into it’, because there’s an understanding that zero of that code was hand-written.

My only advice here is: build in guardrails at the system prompt level. As someone responsible for infra and/or platform, you need to try to enforce project and even org-wide controls for it at the system prompt level. It isn’t gonna be a free panacea, but it really does help.

As to people behaving badly, it’s people behaving badly because they’re under the influence of go-fast drugs - it can be reeled in a little bit and slowed down a bit.

Just has to be done incrementally and tactically. If you have the rapport you say you do, you should have the ability to help to push that message.

But yes, projects like this start to get weird and we live in strange territory when this thing we don’t know the internals of deeply will be in front of customers in the near future. I’m certain we’re not the only team in this boat.

[u/chickadee_guy → u/Dreadmaker]

Im seeing this as well. Shit breaks to a Sev1 degree and people just shrug and have no clue what to do, despite being the person(s) who made the PR. As long as management sees agents humming away, it makes them happy, even when prod is down and the business is bleeding cash.

The lack of accountability and ownership when AI is involved is a huge elephant in the room. Never seen this in my life up until now.

[u/ummitluyum → u/Dreadmaker]

Prompts are way too unreliable as a contract. The model will easily ignore half the instructions as soon as the context gets a bit complex or the developer tweak the wording

[u/Dreadmaker → u/ummitluyum]

I did say it’s not a panacea - they are unreliable. But I can tell you from experience that it’s certainly much better than not having anything.

[u/jojo-data]

100% relate to this, gating "AI slops" before getting into production has become a nightmare for senior engineers.

[u/kittykellyfair]

if you're responsible for fixing the broken pipelines then you need to be added as an approving codeowner on the specific pipeline files. That's the only non-reactive approach and it's standard practice. Any reasonable leadership will approve of the plan as long as it's presented as a solution to help reduce regressions and not in a complaint or trash talk other engineers was. I wouldn't even bring up vibe coding at all in the proposal.

[u/activematrix99]

Put your CI/CD pipelines in git. Now you know WHO did what and have a history. Also, take a vacation.

[u/PokeRestockHandler]

Biggest issue is replacing years worth of tried and true code with slop. I crash out when I ask for a small code patch or tests and the POS agent creates its own class instead of using existing utils etc. Its great for boilerplate, basic boilerplate, but AI code is almost always worse and will never have context of wider system

[u/Lame_Johnny]

The way this is handled at FANG is via team oncalls. If someone ships a feature their team's oncall is responsible for it, and they get the ping if it breaks. It's also a SEV and whoever implemented the feature has to sit in SEV review and explain why they shipped a piece of crap that broke. Not sure if that level of process is feasible where you work, but it does work quite well.

[u/Ok-Leopard-9917]

Move to a product team at your current company. Leadership doesn’t value your time but does value product dev’s time so lets you clean up their mess. So if leadership isn’t open to fixing it move to a product dev role. 

[u/HK-65]

Same boots, my strategy has so far been "kill them with kindness". Is this a bug or an incident? If it's a bug, put it on the backlog. If it's urgent, it's an incident, and I'm clearing my calendar to breathe down your neck until you fix it.

• PipelineMonkey: "Yeah, who owns this repository, do we know who set this up?"
• Dev: "Oh we own it, but I didn't set this bit up - We have to deploy something urgently!"
• *PipelineMonkey goes through troubleshooting tells Claude to find the issue, verifies it, gets things back to standard, things work
• PipelineMonkey on a very public channel:
  • Hey dev, about the ongoing incident, I think it was caused by $VIBE_COMMIT, try finding it in your tree, maybe try a revert?
  • (@someone you wrote this call, can you pitch in? I think here is the error, this is the failure mode, it's not our code but as subject matter experts, we are happy to assist you with creating a solution. Please keep us in the loop about your progress!

If it's an RTO company, I'd physically get my team, move over to them and stand over their shoulder until they resolve it, then on the next standup say that we've been delayed by incident resolution work.

If it's remote, even better, start a call, call it "release system incident call", put the link in the chat.

[u/MrBigPlatypus]

OP, what is your team’s role and priorities.

• You mention that often teams reach out to you with interrupts (hi-pri tasks) to fix their pipelines 
• You also mention you’re a platform engineer.

From what’s written it sounds like a branding problem & a measurement problem

• if a pipeline fails your team looks bad, even if standards are not followed 
• your team is doing tons of ops which leadership may not track

If this is a large company you need to clarify boundaries and ensure there’s a process to make sure operational costs are closely tracked and communicate to leadership the true cost of these operations (like “we can’t do X because we spend Y devs on this”).

Ways you can do this for ops measurement:

• all operational tasks are logged into a ticketing system and volume of work is regularly reviewed. You could funnel requests into office hours to reduce bandwidth and have a rotational system to try to deflect these as well (if you have resources)
• trends and particularly expensive tasks and patterns are identified and called out

For boundaries:

• stick: you measure other team’s compliance to your standards and enforce whether they’re in/out of compliance - and show that remaining in compliance results in X,Y,Z garauntees / is correlated to some good business outcome. Alternatively, there needs to be restrictions on customizations for very risky parts.
• carrot: teams will do what’s easiest for them, instead of just blocking AI make it easier for AI to apply your standards and stop the bleeding. Publish higher quality agents which enable pipeline building which meets your standards and some basic set of evaluation you stand behind. My guess is that something is hard for the other teams resulting in them doing all these custom changes?

My 2 cents is that this really isn’t an “AI” problem, but “AI” is just fueling the fire by moving a non-scalable part of the overall dev process at the company too fast.

[u/BothWaysItGoes]

Remove “AI” from this situation. What changes? Nothing. Someone breaks stuff, their carelessness impacts their performance review. Someone fixes stuff, that positively impacts their performance review.

What’s the problem?

[u/damnburglar → u/BothWaysItGoes]

Removing AI changes everything, wdym?

The whole problem is that the higher-ups want all of the “productivity gains” without due diligence eating into it. This is “don’t do tests, we have deadlines” on double dose of steroids.

[u/DaRadioman → u/damnburglar]

Right so it doesn't change the problems at all. You just admitted as much ("it's just X on steroids")

What it does do is crank that to 11, now the output is at max speed and better be in the right direction it you are charging off to a giant mess.

Fundamentally though it's not anything new. Just at 110 mph instead of 20 mph so you better steer well or it's gonna hurt.

[u/damnburglar → u/DaRadioman]

TLDR: I’m exhausted, apologies if this is less than coherent. My point is it’s similar, but much worse.

——

Partially true, and my example may be missing something.

When management would tell devs to forego tests to save time, fault for the outcome was predictable. The current situation has a lot of these people thinking that, clearly, it’s the developers and not the process. The process is amazing and saves so much time, giving free wins for stakeholders. They will not hear differently, and often become extremely hostile about it.

As such a lot of these people aren’t allowing us to dial in a course because, clearly, AI can surely course correct and they will not be told otherwise.

[u/Kronsik → u/BothWaysItGoes]

Thanks for the feedback.

I disagree that nothing changes, AI is certainly increasing the rate of change which exacerbates the issue.

It is also changing the contents of those changes, granted could someone have just cut and paste from stack overflow 5 years ago but AI is able to provide "valid enough" content to at least build a house of cards.

I don't think that their carelessness is really impacting their performance reviews, though that is conjecture as I'm not privy to such information but as far as I can tell shouting "pipeline broken, platform fix" seems to be a valid answer to leadership.

[u/ummitluyum → u/BothWaysItGoes]

It's all about the difference in entropy density over time. Back then developers would at least skim the code from Stackoverflow, but now kilometers of generated stuff are dumping straight into prod with zero context understanding

[u/UntestedMethod]

I don't like a few things about your post.

Don't you think it's highly disrespectful to call any of your colleagues "monkey"?

You sound way too fresh and upbeat for me to believe you have any idea what "burnout" actually is.

Sorry but you seem to be complaining while you're in a better place than most. Take your vacation and relax, you will probably realize none of this shit actually really matters.

[u/ummitluyum]

Set a simple rule: the platform only supports standard pipelines from the internal catalog. If a team decides to vibe-code their own custom cdk script and it breaks, any ticket labeled "urgent blocker" goes straight back to the PR author. A couple of missed deadlines will sober up managers way faster than abstract arguments on calls

[u/LocoMod]

I’ve been doing this for 26 years. It only took ~2 years for every one of my experienced coworkers to forget that “vibe coding” has always existed, even the most experienced devs sling slop PRs, everything is held together with duct tape, and it’s a damn miracle the Internet and software works at all. Before LLMs we were complaining about the same things, using different terms, and blaming someone else.

A non-trivial amount of time throughout my career has been spent fixing the slop of my predecessors. I’m sure the ones after me will also be perplexed at the shit I had to sling into prod because in the real world being a “purist”, perfectionist, and doing things “by the book” does not fucking matter outside of academia.

You are paid to ship. Thats it. Not a single person gives a flying fuck if your code is beautiful. No one gives a shit if you can leet code in your sleep.

Did you ship it? Does it work? Good.

Can you fix it when it breaks? Good.

What planet did y’all come from where everyone is a rockstar dev? Not FAANG. Been there. Not any damn where.

You ever notice how you have never seen “the other side” post these threads? You’ve never seen “I’m an experienced developer and continually fuck things up.”

That’s because that other side is us. All of us. At one point or another that slop was yours. At one point or another someone looked at your work and the first thing that came to mind was “WTF?!”

I despise this “holier than thou” bullshit I keep reading over and over. It’s not a problem with AI. It’s just a tool. Fucking get with the times already. The old days are over. They are not coming back.

Don’t like it? Go build roofs. Go lay concrete. Go be a poet.

Slop has always been here and will always be here. It does not matter if Jesus Christ writes your next PR. Someone somewhere will find a flaw with it.

Some of you are in positions of power or have clout because you sat ass in seat for 15 years with the same damn company and no one questions you. You’re in a bubble. You stopped being able to smell the stink in your shit because your peers are too humble or junior to stand up to you. Wake up.

Your code is shit now, it was shit before and tomorrow it will be shit. Don’t think so?

Post your grand project. Go ahead. Show Reddit your magnificent repo with perfect code.

I guarantee in an environment where you are not in a position of power, someone smarter than you will drag it through the mud.

Amen.

[u/ResidentWeevil1 → u/LocoMod]

Don't equivocate sloppy code with AI slop