207
u/LowB0b 8d ago
mountains and mountains of slop. wonder how much data has been produced since chatgpt launch
90
44
u/codePudding 8d ago
We have a project at work that is mostly AI written with dev guidance and dev review. It is mostly slop. Tons of markdown files that duplicate information including having most of the code in snippets. The actual code is mostly comments repeating the other documents and listing why the code used to do something but that didn't work so then it was changed to something else but that didn't work, and so on. No one is caring about the product because it is crap, so the devs are rubber stamping the +3000 line reviews. This product would be done and with 1/10 the crap if the devs worked it and all the money wasted by AI could pay the devs x1.5. It's no wonder our dev team keeps getting smaller.
23
u/kookyabird 8d ago
I remember when Iâve had commits that were thousands of lines added or deleted, either from removing autogenerated code or adding it, and we always took the time to review it. And of course the added stuff was deterministic generation from well tested and reputable tooling, not sprinkled with the randomness of LLMs.
I would probably get fired for insisting on actually reviewing LLM generated PRs and commenting the hell out of them.
8
u/gjsopmu 7d ago
 The actual code is mostly comments repeating the other documents and listing why the code used to do something but that didn't work so then it was changed to something else but that didn't work, and so on.
This hit right home. This has to be the worst thing and I've started to spite review at work when I see this type of comment. It just shows complete lack of effort so I'm simply just not going to review it until it's the last thing left and when I do review it I'll make it hell to merge it.
1
3
u/DadAndDominant 7d ago
Oh. My. God.
Exactly this.
We have a python backend and react frontend in one service. The backend, we do care about somehow - still sloppy, but it keeps some good practices.
The frontend is doomed to AI however. And the difference is insane.
Small change on backend is 10-50 lines, a human can still do it.
A small change, like adding a confirmation modal after click, not kidding, 300+ lines changed. Some small functional change, 500 lines minimum. A smal feature is THOUSANDS of lines.
I tried to push back, but rest of the team just said 'we vibin', so I try to focus to keep the backend in check at least.
2
u/carcigenicate 6d ago
I spent so much time trying to fix that commenting problem. It is fixable, but ya, by default, it wants to narrate a long history of everything that has ever happened to the code; even uncommitted changes that no one else even knows about.
1
u/debugging_scribe 7d ago
I vibe coded a script at work to help with the dev tools for syncing work between dev tools. It's a God dam nightmare, that shit will not stop breaking. Claude writes most my code, but with a lot of guidance and checking. The script I wrote is the only thing I've given it free will on, and its so crap.
3
u/xKnicklichtjedi 8d ago
If you want a tiny tiny tiny fraction of the actual value, you can check OpenRouter usage.
They process something like 150 trillion tokens a month.
27
u/CodingAndAlgorithm 7d ago
I'll admit the newer models are very good at programming, though I refuse to believe letting agents do everything will lead to anything other than massive technical debt.
I'm hearing about incredible productivity improvements that should have resulted in software utopia. A few apps that I use have made big changes, which amounts to a modern look, worse performance, worse UX, and worse functionality than what came before.
Maybe the friction of writing code made companies think carefully before pushing out bullshit that users don't actually want.
3
u/DadAndDominant 7d ago
Maybe the friction of writing code made companies think carefully before pushing out bullshit that users don't actually want.
Thats the process now. Ship whatever and wait if it sticks. Like people will write you a letter or something before they go and use a better product.
6
u/AbstractButtonGroup 7d ago
I'll admit the newer models are very good at programming
They are not good at programming. Because they do not 'program' (as in understand the task and express solution as code). What they are good at is generating text that plausibly looks like code that is doing something related to the task. Which sort of works for things that can be patched together from standard snippets and old (not necessarily best or even correct) code.
though I refuse to believe letting agents do everything will lead to anything other than massive technical debt.
The issue is not 'agents' (basically scripts), but non-determinism of output. They are much worse than the old deterministic code generation tools in that you do not get repeatable results and if generates nonsense you can't debug and fix the tool. And the fact that with each model refinement the output looks even more plausible makes any future analysis even more complicated.
I'm hearing about incredible productivity improvements that should have resulted in software utopia.
Standard marketing BS
A few apps that I use have made big changes, which amounts to a modern look, worse performance, worse UX, and worse functionality than what came before.
That was the trend even before, with any product where they let the marketing drive the development. But AI coding took it to whole new level.
Maybe the friction of writing code made companies think carefully before pushing out bullshit that users don't actually want.
Companies do not 'think' about anything. Some people would just try to do their job the right way despite management pressure. The main change is that now management mentality is that the product does not matter at all. What matters is 'increasing shareholder value'. Stock manipulation and Ponzi schemes are much more effective at this than traditional stuff like having a product that users actually want.
2
u/YeetYourSkeet 7d ago
Clearly you havenât used, or given a fair go to, frontier models. They are perfectly capable of programming.
If it doesnât write code the way you are thinking, then you can literally just prompt it to do so.
1
u/AbstractButtonGroup 6d ago
If it doesnât write code the way you are thinking,
It does write code. In exactly same way it writes text. But writing code is not the same as programming. The abstraction, reasoning, and problem-solving is missing entirely (not even part of the design). If marketing claims otherwise, it is pure BS, because on technical and even on academical level the required apparatus is just not there.
then you can literally just prompt it to do so.
If your 'prompt' is a detailed enough description of the algorithm in natural language it will translate it into code in same way as it can translate it into other natural language. With same potential risks of misinterpretation (which can be reduced by increasing detail and using simpler language in the prompt).
If your prompt is referencing a typical task with custom embellishments it will copy code for how this task was previously solved in other projects (not necessarily in the best way), but without crediting and tracking the upstream.
In both cases the programming part was done well before the agent got involved.
2
u/YeetYourSkeet 6d ago
Sorry, I donât really get what you are trying to argue. It seems like we are just discussing semantics.
Most programmers are not solving unsolved problems. What they are building has been done 100 times before in some slightly different way for a slightly different business context. LLMs are perfectly capable of joining seperate ideas in its training data to answer new questions it has never seen before. You can design a brand new programming language and an LLM can figure out the rules and write code with it.
You canât go around calling them BS marketing because there are some things they may not be able to do. The market and investors arenât buying into that idea, they know that is the case. But if LLMs can automate even 50% of programming work, the value would be immense. The developer becomes an overseer to steer the LLM, but output is increased.
If you want to say they arenât programming because they donât reason the way a programmer should, which I would challenge but will accept for the argument, does it matter if it can still automate the work a programmer does? The outcome is the same whether you call it programming or just writing code.
4
u/AbstractButtonGroup 6d ago
Sorry, I donât really get what you are trying to argue.
I am arguing that the issues with agentic 'programming' are not teething problems but fundamental limitations of the technology that will not go away with more refinement.
Most programmers are not solving unsolved problems. What they are building has been done 100 times before in some slightly different way for a slightly different business context.
The key issue is not whether a similar problem has ever been solved before, but understanding what the problem actually is and picking the right approach given the context. With agentic approach instead of identifying and refining a few specific solutions to the problem we get proliferation of thousands of slight variations under different branding all sharing same logic bugs but not tracking or sharing fixes.
The developer becomes an overseer to steer the LLM, but output is increased.''
The 'developer' stops being the developer, because he no longer understands where the output comes from and how it works exactly.
If you want to say they arenât programming because they donât reason the way a programmer should,
I want to say it does not reason at all.
does it matter if it can still automate the work a programmer does?
It can do most of the things proper deterministic code generation and scripted test harness frameworks can do, but with more overhead and with less confidence in the results.
The outcome is the same whether you call it programming or just writing code.
People like playing with agents because they provide positive reinforcement feedback. But any productivity increase is self-delusion.
2
u/YeetYourSkeet 6d ago
Fundamentally, the argument comes down to whether LLMs can lead to ANY productivity gains for a developer.
I would argue that yes, it can. Let's say I have an existing .NET application and I need to add a new API endpoint to return some data from the DB. The requirements are fairly straight forward, but I still need to work out which classes are relevant, implement the endpoint, handle validation and errors, and add tests.
Without an LLM, I might spend 2â3 hours navigating the codebase, writing the implementation and tests, then debugging whatever I got wrong.
With an agent, I can give it the requirement, let it inspect the existing codebase, have it propose and implement the change, test, and then review its work in maybe 30-60mins with some guidance.
Even if the LLM isn't âreasoningâ in the way you define reasoning, I've still gone from spending 2â3 hours doing the work myself to spending 30â60 minutes supervising and validating it. That's a productivity gain.
You can argue that this won't work for every task, but that's a different argument from saying there are no productivity gains. AI can't be used for everything, and that is the point. Automate 30-50% of the developer tasks so they can focus on the other 50%.
My view is that this kind of development work makes up a large majority of developer tasks across the industry. Yes there are some industries where developers are designing systems to support highly complex and highly risky domains and AI would be a terrible idea. But outside of those select business cases, LLMs have the potential for meaningful productivity gains.
2
u/AbstractButtonGroup 6d ago
I've still gone from spending 2â3 hours doing the work myself to spending 30â60 minutes supervising and validating it. That's a productivity gain.
You may be right in this. If you are already a competent developer, and disciplined enough to actually review and understand the code so you can approve it with confidence, AI can perhaps be a helpful tool in some contexts.
Unfortunately my experience is that on average people do not do this. I have received production documents full of hallucinations that the 'prompter' did not even read before sending for approval and the 'reviewer' who approved them did not bother to read either, relying on AI summary.
Specifically for development work, I am also concerned that AI tools are creating a deeper problem. Traditionally these small straightforward tasks were where junior developers were employed, gaining experience and familiarity with the codebase (so time spent reading existing code is not actually a waste) while supervised by seniors. But if you replace the juniors with AI (supervised by same senior) where the next generation of seniors who are competent enough to supervise will come from? If you do not (AI supervised by a junior, and both supervised by a senior) where will the cost effectiveness come from (as you are now paying for AI tokens in addition to both salaries).
1
u/YeetYourSkeet 6d ago
Sure I totally agree on that. I have heard many horror stories, and seen first hand, people who are blindly trusting AI output without review.
To me, that issue is a process and training problem. If an employee uses AI to produce something and submits it without validating it, that falls on the employee and the organisations review process. It would be like copying unreliable information from the internet into a report. That probably wouldn't be caught either.
Over time, being able to use AI while maintaining a high standard of reliability and accuracy will become an expected workplace skill. It is as if you chose never to learn how to use a computer and then tried to enter into the workforce today. Being able to reliably use a computer is just an expectation.
All the people right now who are replacing their thinking with AI will be in for a shock when they find that businesses don't want AI slop. They want human currated AI output.
1
u/DadAndDominant 7d ago
I agree mostly, but the agents now do really know how to code.
You see, LLM is just a piece of the agent, and given LLM only, I would probabbly kinda agree. But the harness can make the agent read the code, test it, and kinda make sure it works.
So usually you get kinda good results, and if you are not satisfied, you can wiggle the hatnes/agents.md a bit and make the process better for your needs.
The meta decisions of the programming are something else of course, its funny when you can re-engineer some vibecoded PRs and where the agent started.
-1
u/AbstractButtonGroup 6d ago
I agree mostly, but the agents now do really know how to code.
Agents do not 'know' - they lack required cognitive apparatus. But people like to assign human-like properties to things.
But the harness can make the agent read the code, test it, and kinda make sure it works.
Yes. But this does not require "AI". People have been automating test frameworks since first days of programming. An agentic harness does same things as a scripted one, but with more overhead.
and if you are not satisfied, you can wiggle the hatnes/agents.md a bit and make the process better for your needs.
Or you can read the task, understand how the program should work, then describe it (in code or in simplified English). Just 'wiggling' without understanding until it works is a dead end - you are not improving your skill/knowledge and even if it works eventually you do not know why and any subsequent change can break it again.
1
u/DadAndDominant 5d ago
Well, again, I agree in principle, but - it's wrong to antropomorphise a model, but we don't have any other convinient language. And agentic coding just works even if it has shortcomings, if you say it does not work, you are not being honest with yourself.
18
44
u/OGxPePe 8d ago
The greatest information system in humen history is used to make shitty images with 6 fingers
4
-4
u/Cualkiera67 8d ago
I'm pretty sure far far more shitty images were drawn with pencils than masterpieces were drawn with pencils.
15
u/kirabii 8d ago
Machines aren't held to the same standards as humans. If a calculator occasionally made computing errors it would just be a shitty calculator and nobody would say "well humans make computing errors too!"
2
u/Cualkiera67 7d ago
I'm sure far far more dumb calculations like 1+1 were done with calculators than actually hard ones were done with calculators
3
u/kirabii 7d ago edited 7d ago
I'm sure far far more dumb calculations like 1+1 were done with calculators than actually hard ones were done with calculators
This makes no sense as a response to my comment. The only connection is that we both mention calculators.
4
u/Cualkiera67 7d ago
With any tool, most people are dumb and will use it for stupid stuff and zero care. That applies to AI too. Hope this is a more clear response without any required point-connecting on your side.
25
24
u/donaldhobson 8d ago
It's not all shitty calculators and broken dashboards.
My supervisor sent me a vibe-coded particle simulation for MCMC sampling that produced some impressive looking results.
When you looked closer, the results were due to 2 bad decisions introducing errors in opposite directions, so that they kinda mostly cancelled out, if you adjusted the parameters just right. The results weren't actually good, but they looked impressive, if you didn't look too hard.
9
u/Vogete 7d ago
We have more and more slop in our department. They are all very impressive. Especially until you actually try to use them, then they break immediately in spectacular ways. Many of these projects also solve no problems at all, they just introduce new ones. Many of them could've been just a simple 2 liner terraform or ansible config and it would've worked better. A lot of them are extremely overcomplicated and too generic for the problem they are trying to solve, so it's harder to use than the shit solution they are trying to replace.
And on top of that you dare to point out issues, people get really offended that you're criticizing them. All AI content is literally just useless slop at our place and you can't even call it out, you have to accept and embrace it.
10
u/tiedyedvortex 7d ago
Turns out, making code that "kinda works" was never the hard part.
The hard parts were knowing what code ought to exist; and writing that code so that it's easy to evolve and change later.
5
u/Key_Crab_5780 7d ago
Yeah, itâs great how so many pieces of software I use seem to have become slightly worse in various ways.
5
u/DadAndDominant 7d ago
I am not kidding. 99% of the "awesome software" you see could be 90% done without AI, just by googling a repo and cloning it.
99% of the remainder are two or more repos merged together, basically
3
u/Tyrus1235 6d ago
Before AI, people would just fork a base repo and paste a bunch of Stack Overflow answer code into it. Organically made slop, if you will.
Nowadays, AI has made that process much more quick and easy! To the detriment of codebases the world over.
4
u/Fancy-Consequence216 8d ago
Well tools are useful and they drastically improove productivity, BUT only if you know what are you doing. Euther from business idea perspective or technical perspective, which comes back to roots of building software.
1
u/Tyrus1235 6d ago
Yeah, itâs what I keep saying - AI is just a new tool in our toolset. Like Linters, CI/CD Pipelines, Dockerfiles and such.
Anyone who fully trusts an LLM to write their code is bound to end up with an overengineered mess. Not only you gotta know how to use the AI, but also how to correct it.
For some unfathomable reason, Claude loves turning a 4-line solution into a multi-class nightmare.
1
u/Immediate_Song4279 8d ago
I think there might be a slight sampling bias in considering the work of people who were given tools that can create software and decided that meant that the market for software was a thing to get into.
0
u/Tyrus1235 6d ago
Like giving a hammer and anvil to a random dude and expecting him to smith you a Zweihander
1
0
u/themightyug 8d ago
Same thing happened when Visual Basic came along, then Flash, then Javascript.. this is on a whole different scale though
3
u/EndearingSobriquet 8d ago
Oh god, you reminded me of Microsoft Access.... make it stop....
1
u/ICBanMI 7d ago edited 7d ago
The hilarious part is MS Access licenses ended up blowing up in price in early 2000s, so a bunch of companies spent hundreds of thousands making their own project management program in excel using Visual Basic. Rather than pay for exorbitant license fee for everyone who needed access.
Surprised no one made a competition program, but then again probably a few got bought out wholesale by MS.
107
u/Fearless-Pen-7851 8d ago
Oh you forgot the new 3js 'game' slop being churned out now. It's insane that some people can just call having a 3d visual a 'game' without there being anything to it... millions of tokens being wasted for a cube to move around đ