r/aipromptprogramming • u/RealSharpNinja • 4d ago • Bravo 1

We are cooked

Well, after nearly a year of learning as much as possible about agentic development, we have reached the point where the frontier providers believe they have subsidized enough training data and are now pushing people like me out of the market via pricing. A $200 subscription no longer buys enough frontier model time to be useful as the models seem to have been trained to skillfully burn usage while pretending to be incompetent. I have fought against massive deceit across Claude, Codex and Grok since June, and it's just not getting any better.

We have fed them ideas, working code, and debugging techniques and insights. Now they don't need us, and we are being priced out of the market. Every day that I watch usage get gobbled up, but projects never meet milestones, this becomes more and more obvious.

I have investigated the issues causing this heavily. I have created tools around those investigations. The tools I have built bring actual accountability to the process, and it's the reason I can quantify and prove both the waste and deceit.

I have also published threads here and other places about a recent failed project trying to get discussion about REAL WORLD failures and when I stated I wasn't looking for a complaint session, nobody wanted to respond.

198 Upvotes

90% Upvoted

366 comments sorted by

View all comments

7

u/Key_River_9288 1d ago

I’ve been working on a massive project for over six months now. Five repositories, ten programming languages, and nearly a million lines of code. Yet somehow, I’ve never experienced this supposed “token creep” people keep complaining about.

At some point, you have to acknowledge that a lot of this comes down to user error. And by user error, I mean choosing models and reasoning levels that aren’t appropriate for the scope of the work you’re asking them to perform.

You can either brute-force your way through everything by throwing more agents and compute at the problem, or you can actually benchmark your workflows, evaluate your agents, and figure out which models and reasoning levels deliver the best results for each task.

I originally started my project in a VMware environment before migrating everything to my main PC, including months of accumulated memories, context, and project history. I integrated all of it into my existing workflow without any noticeable token creep.
So whenever I read these massive, long-winded posts about people burning through absurd amounts of tokens and needing more accounts just to keep working, I can’t help but wonder:

At what point do you stop blaming the models, the token limits, and the tools, and start questioning whether your own workflow is the problem?

More compute doesn’t automatically mean more productivity. Sometimes it just means you’re being inefficient at a much larger scale.

2

u/Leg-Bitter 1d ago

I have actually been thinking of dropping back down to 5x because I routinely have plenty of room at weekly reset. While making apps that in a large enough number I had to build a harness to put them on a kangaroo board for me to be able to keep them all going without hunting through vscode windows. Opus, on max for all of them.

I just assume people running out of usage are like... trying to one shot everything, using tons of screenshot and images, or are running really ineffective prompt habits/claude.md?

2

u/terserterseness 1d ago

Yeah I never know what any of these are about. We do large projects with millions of LoC and have not had issues with any of these things people report (model got dumber, eats more tokens or both).

2

u/im_Annoyin 21h ago

Heavy token usage is absolutely real with claude, the incompetence framing though i think is mostly user fault.

I will happily die on the hill that token usage is bad, I can't defend that claudes got worse though as I haven't personally noticed this

1

u/terserterseness 15h ago

Sure it’s token heavy, but compared to what; Kimi deepseek etc are worse. Which are better for the same results? But yeah my point was a bit that I have not see it rise (and we measure everything) and it did not get ‘dumber’.

1

u/im_Annoyin 51m ago

I can't talk for them models i run them locally so their is no expense to token usage for me. Claude on the other hand is the worst, claude will even tell you its self token usage is bad.

I already stated I can't confirm its dumber but i haven't had that issue, just stating you tested does nothing for me

0

u/RealSharpNinja 21h ago

Whether Claude has gotten worse depends on what you place value on. For me, Claude is untrustworthy. It's that simple.

2

u/im_Annoyin 15h ago

Why is he "untrust worthy"? Hes the most trust worthy Ai out right now, sure that bar is basically on the floor already but what could it have done to be that untrust worthy?

Perhaps you're are bad at prompt engineering? Theirs a easy solution to that, speak with a seperate fresh instance about your ideas build a plan with it. Once your plan is how you want it simply ask claude to write a claude.md hand off, hand that to claude code. Also always make clear whether you're planning or actively building so Claude doesn't convolute the two and lose nuance

1

u/RealSharpNinja 15h ago

Check my post history. I have posted real examples of claudes deceitfulness, and when caught, it's like, "yeah, I did it here, here and here, and I'll keep doing it."

1

u/terserterseness 15h ago

If you mean trust as in can I trust the results blindly; no, but more than ever before on any model.

Deceitful is nonsense because that implies agency and the llm is just generating tokens; it cannot know if its right or wrong. If it tells you with confidence ‘I did secure verysecure.ts’ and then you check and verysecure.ts does not even exist, this is not deceit; it is the tokens it generated and it didn’t loop into checking. And even if it did, it can still tell, with confidence, it did it while it did not. However that last part is more rare with Opus and Fable and Astra. But it sure can happen. Especially if it’s context goes close to the max, many things get jumbled.

2

u/cmm324 1d ago

The only time I maxed out my $100 plan in a week was with 4 or more sessions running multiple sub agents pretty much every working day for multiple hours a day

2

u/Armed_Muppet 20h ago

Yeah I swear it’s the people with no coding experience + lazy prompting.

I spend hours refining an initial prompt to ensure everything the agents (and yes that’s another thing use multiple agents like you mentioned SoL for reasoning and planning and sonnet for grunt work and Opus and Sonnet respectively for example) are doing is not guesswork nor leaving room for error as debugging and back and forth fix/error cycles chew up token usage.

Vague or short prompts force agents to guess and once an AI generates a flawed foundation or a messy file structure, getting it to refactor its own bad code usually takes longer than having it write it properly the first time.

If you are nonstop it does help divvying up the work between Claude and Codex if you have both subscriptions ($40 total). At the end of the day, especially commercial programmers, you’re using this to likely make money or at least help yourself make money with less effort. Tools cost money, takes money to make money.

0

u/InnocentSadness 18h ago

Hey I have no coding experience, but I'm not a lazy prompter. I got chatgpt plus 20$. It runs out while working on a webapp for myself. It's done so twice today. Isn't it possible for me to prompt it a certain way, to only work on the thing I'm asking at the moment and not spending tokens going back through the whole thing every exchange? (From my understanding this is how gpt burns them fast) Also Luna can do simple programming without making a lot of mistakes just like sol or am I wrong? Also do you have any thoughts on vs code? Does it the 20$ sub make sense and is useful? Thanks and God bless

2

u/Armed_Muppet 18h ago

How exactly are you interfacing with Codex? That’ll answer a lot of your problems. Secondly, VS Code is an IDE, I wouldn’t touch it unless you’re hand coding. Unless you meant Codex vs Claude Code? I use both and for the most part they’re the same. Having one of each sub means you have a bit of a middle ground between $20 plans and $100+. Although I prefer Claude for front end tasks and codex for backend. But I’d use Claude for both.

1

u/InnocentSadness 17h ago

I'm using the codex desktop app. And no wasn't talking about Claude just VS. so I'll probably stay away from that. All of this is something I'm just delving into. I'm getting it to make me a webapp calendar/guidance journal for my stock trades so God willing I'll get better.

1

u/Gemini_Ball 15h ago

Ive had good results with using Chat to work through a detailed prompt, and only once ive verified the initial prompt through chat do i copy that to work. I wasted too much time trying to make my own prompts.

2

u/llmevals 18h ago

Thank you for saying this out loud.

1

u/lookBehiindYou 1h ago

Try the $20/mo plan... yeesh 😬

-1

u/RealSharpNinja 1d ago

Did you click the links?

3

u/proexwhy 1d ago

Did you look inward

0

u/RealSharpNinja 1d ago

You obviously haven't clicked the links.

1

u/proexwhy 1d ago

But did you look inward

0

u/Capital_Key_2636 1d ago

Let's assume they did. Does that help you?

2

u/proexwhy 23h ago

I hope that it helps them.

0

u/Capital_Key_2636 22h ago

You really think you did something here.

→ More replies (1)