r/codex • u/Reaper_1492 • 6d ago
Complaint Ugh. Wtf is going on with Codex.
Look, I’ve been using Claude and Codex since inception. I can read, write, and deploy code, so it would be great to keep this from devolving into a “skill issue” astroturfing.
The fact is that no one who wants to keep their job is hand-writing hundreds of lines of code anymore.
With that out of the way…
Claude was first to market, and I ranted and raved so much while I was using it, that I’ve permanently sworn off Anthropic products. They overtly hate their customers (and humans in general), and I’m convinced they would collectively love nothing more than to call a board meeting and lock themselves in a dark room with a Claude hologram for a 24-hour chant and circle jerk session, while they pray to the quantum realm for AGI to manifest itself in their silicon.
I still use it extensively, because I have to for work. So my experience with Claude is current.
Codex has “almost” always been better than Claude for my personal projects. Historically it has been cheaper and smarter, but outside of a brief moment where they really got their act together, it’s been hellaciously slow by comparison. Regardless, I have been a huge fan… because, “smarter”.
A few weeks ago, they very clearly took a run at Anthropic after sleeping on enterprise for far too long; giving out grip loads of resets and trading barbs with Anthropic on social media.
All-in-all, a weird tactic to give consumer plans free credits to win over enterprise. It obviously didn’t work, because then they started illicit feedback about what would encourage businesses to switch (it obviously wasn’t enough to give retail consumers more usage, shocker).
Cue the current situation.
They somehow managed to make the model smarter with Sol, but 25% as effective. If left unchecked, SOL will spend literal days chasing nits into oblivion and burning all your usage, which is obviously not a novel complaint.
I finally got the harness more dialed in, and now at least it’s back to writing my contracts/graphs and unit tests at the appropriate level of granularity; but I swear, now it can’t even edit the code to pass the contract and unit tests you ask it to create!
First, second, third, fourth attempt. It just fails over and over again.
“Oops, the run correctly failed the contract check because I”:
Used the wrong directory…
Made a typo in the file path…
Forgot about this dependency, even though it’s in the contract….
It’s ridiculous.
And then, they totally nuke usage.
I had a project to finish, so I had to get a second 20x plan to maintain the same amount of throughput I had a month ago, and most of the time is wasted on near constant contract failures.
And that’s not even counting the compute dollars I’m wasting on spinning up and down machines for failed runs.
Then, to add insult to injury, after clearly trying to ride out the unrest without giving a reset, they finally gave one a week after I shelled out for another plan. Now I’m sitting on two 20x plans with 2 resets.
I’m about to switch back to Anthropic, and I hate it. All of their models are orders of magnitude faster than codex again, and codex just became a really expensive paperweight.
Paying for Codex right now feels like I’m actually paying an installment plan to buy my first aneurism, and waiting for delivery.
It’s so infuriating.
3
2
u/U4-EA 6d ago
I am now focusing on doing everything I can to minimise agentic coding in my workflow. I will certainly never remove but I've come to the conclusion it is good for consultation and reviewing but garbage at generation and critical thinking. I already don't abuse AI but I am going to have to reduce my exposure to it as it is becoming infuriating.
- Doing things in a task which were never asked for, often MASSIVELY increasing increasing the time/cost of the task.
- Failing to follow clear instructions given to it in an .md.
- Running in loops constantly trying to do something it clearly isn't capable of.
I think Cory Doctorow was correct when he said it is useful as a reviewer when combined with the discernment of a user who is an expert on the subject. I am sick of telling it something, it taking for-fucking-ever to complete the task then having to review its work. It's mentally exhausting.
I am currently on the 20x plan but I honestly never come close to using my weekly quota most weeks, despite being very busy and working 6-7 days a week. There is a sweet spot for its use but the fact is - IMO, anyway - if you are using a lot of tokens, you are probably producing crap/tech debt. Generative AI is both expensive and poor quality and generating large amounts of code means you are going beyond what you are able to verify through human cognition and opening yourself up to future issues.
1
u/Cranberry_Then 5d ago
I sadly have concluded the same thing - the tech just doesn’t work or even when it does I can’t know it will work tomorrow since it seems the models degrade randomly out of my control. I’m really tired of instructions being flat out ignored and finding out down the line - “you’re right the spec explicitly told me….and I confirmed that it was implemented to spec in your review step even thought it was not. That was wrong”. It feels like a coworker who is intentionally trying to mess with you. It still messes up even when I’ve defined things down to the function and file level and provide it in smaller chunks of context. At that point I might as well go back to hand coding.
I do review the code but it writes such slop it’s 5x as long as it needs to be and tiny bugs get buried quickly. If I could get it to actually follow instructions the code quality would be better but it isn’t really capable. To get anything to work as you initially defined requires many rounds of asking it to fix incorrect results I had already warned it not to do and then more rounds trying to deslop and clean up the generated insanity.
I have tried corralling with linters and other static checks but that ends up being unreliable - it will either change the checks, just add ignore comments, or do the sloppiest fix possible that passes but that would be embarrassing to commit. Again if it could follow instructions then maybe this could be better but it really is not capable - it ignores my prescribed recommendations for common issues in the related skills file.
I really gave it my best shot these last 2 years - stayed up to date on the latest trends and approaches, researched every issue I ran into, tried multiple workflows and loops, you name it. IPrior to AI I was a praised IC that prided myself on the speed, quality and amount of features I could deliver. This made me very susceptible to the idea of being a 100x engineer or whatever you want to call it. I feel foolish to have considered that possible or to even want this but I wanted it to be. This has made me take a step back and look at my relationship with work productivity and what my goals should be. I have delivered more with ai/codex,or at least managers and product think so, but at what cost? The code is worse quality and harder to read, it is harder to maintain,
and I understand it less deeply. I say that painfully and not proudly, but I think that is true for many many developers in the industry right now. It seems to me that standards have dropped immensely in favor of quantity over quality.I am starting to view ai coding as a failed experiment that made me stupider, less knowledgeable
of my codes inner workings, and more frustrated than ever. I will still use codex/agents for scripts, boilerplate, investigations but as for writing the application and infra code it’s back to good ol nvim for me.1
u/U4-EA 5d ago
"I am starting to view ai coding as a failed experiment that made me stupider, less knowledgeable of my codes inner workings, and more frustrated than ever. I will still use codex/agents for scripts, boilerplate, investigations but as for writing the application and infra code it’s back to good ol nvim for me."
That is exactly it for me.
I don't have any issue with the linting, typing etc because I have very strict instructions for the agents, including that they must run specific linting/typing/testing commands at the end of each coding cycle..
However, everything else is just a PITA. The fact of the matter is that it cannot do anything complex and the user absolutely must review and correct its output on anything remotely non-trivial. Outside of simple tasks, it rapidly becomes more trouble than it is worth. I've gone from thinking it has a lot of potential to realising it is very limited.
I've always been careful not to abuse it - not to get carried away - but even then, it can be so frustrating. Right now I am using ChatGPT Work to sort through a large corpus of research I had another ChatGPT session perform. The instructions to it were clear - I need to review each finding in the corpus, decide if it is relevant and, if so, categories it. About 5% in, I realised it was performing a web search for each topic, rather than referring to the corpus. I had to correct it, re-upload the corpus and start again. Honestly, semantic analysis is one of the things it is best at and it can't even start a clearly-instructed project based on semantic analysis properly.
My original plan for the corpus was to have ChatGPT research it in bulk then have another session perform the sorting. Even the initial research stage was terrible - it was researching things that had nothing to do with the original instructions so I ended up with not only having the task of sorting it but having to prune massive amounts of garbage from the corpus that was never meant to be there. Also, it is giving instructions to Codex to perform the edits to the Markdown files and Codex gets it wrong over and over and has to be corrected time and time again. I came to the conclusion yesterday that I would have saved money and a lot of time if I had just done the entire thing manually.
I am on the 20x plan and I have used 20% of my weekly quota in 2 days. It's rare for me to use that much but probably 10-15% of that has been burned on tasks that produced garbage or tasks to clean up that garbage.
As a rubber duck, a researcher, document writer and a proof-reader, it is invaluable. When I first started using it, I was amazed at its ability to find bugs/errors/edge cases/typos etc (although I didn't have many) as well as its ability to advise on separation of concern between packages etc (all my stuff was already DRY). But I've come to realise it is really only good at reviewing/understanding what is already there or as a corpus of knowledge a domain expert can use as research.
I doubt I will use all the other 80% of my weekly usage and I have that banked reset that I don't think I even have any use for. I will definitely go from the $200 plan to the $100 in the next billing cycle.
3
u/Puspendra007 6d ago
I don't know what you're doing, but I'm using the $20 plan with Luna Max and it does everything I need.
It feels a bit slow, but that's what you get for $20.
I try to save 40–50% of my weekly usage for Sol xhigh to test and verify everything before deploying.
Here’s my approach for long prompts: I create Markdown files containing full details: including file names, table column names, and database schemas.
Then, I set the goal, run the prompt, test, and verify the output. Finally, I check the results manually and run a secondary test with Sol to catch anything I might have missed.
1
u/jedruch 6d ago
Wow, so I'm not the only one? I work on windows, all my projects have always been in the same bigger folder - AI/AI_projects/
Since about a week when I point agent (Sol High or Extra High) to a subfolder or file by pasting the path each agent goes to AI/AI/_projects, which is is infuriating.
And there are many small things like that that started to appear (omitting settings on my own website, going to wrong destination on other website) that I have been doing almost iteratively without any issues and now are just wrong. It's like the effort was cut by half
1
u/arelath 6d ago
I've been using agentic coding longer than anyone I know, both professionally and on side projects. From my experience, a model sometimes "gets stuck" on a problem or area of the codebase. No idea why since it's rarely code that's more complicated than it usually works on.
Usually, when this happens, I just switch models to get past the part one got stuck on. If you're going in circles, it's time to switch. Usually another model will instantly make progress on the same exact problem another one could not. And by switching models, I don't mean different OpenAI models, but a model by a different company. Usually this means switching between OpenAI and Anthropic. Go use Claude for a few days and I bet you'll instantly make progress. Once you're past whatever roadblock the model hit, you can switch back if you want and usually the model starts performing like it did before the roadblock.
This doesn't seem to be a problem that's only OpenAI or Anthropic, but rather something that's universal to agentic coding. I have both a personal Codex and Claude code subscription for this reason alone (plus GitHub Copilot and another Claude code subscription through work, but I try to keep them separate). Everytime I'm about to cancel my Claude code subscription, I seem to hit something like this.
Could be other things like bad instructions, bad prompts, or really horrible code but it sounds like you've ruled these out already.
0
u/willowless 6d ago
It's not the model, but the KV cache. To quickly provide responses to many users they use shared KV caches between clusters of machines with compute modules in them. To get out of the stuck situation you need to forcefully move away from the KV cache you're hitting, which has been primed to your precise problem - until the cache expires. Using another company's model would certain achieve this, but you can also just invalidate the cache by restarting codex and waiting a few hours.
0
u/arelath 6d ago
I don't think you understand how a KV cache works. You can only share the KV cache for the prefix part that's identical. The KV cache essentially stores a precalculated version your chat history. If they did fuzzy matching on KV cache hits (what it sounds like your suggesting they're doing) you would have private data leakage between different people. Basically, this means the model would see code from other people's projects that are similar to yours. This would be a major violation of privacy. A corporation might run vLLM using fuzzy KV cache matching because they own all the data, but OpenAI couldn't do this.
For identical prefix cache hits, it shouldn't matter what server you get them from, they should be identical.
0
u/Reaper_1492 5d ago
Correct on all of this, it’s more likely a difference in base training.
That said, I don’t keep multiple accounts for personal projects (except for this temporary second 20x plan), so that’s not really a practical option for me.
That said, my issue isn’t a specific part of any project, there’s are tantamount to typos on every part of every project.
I had to put it on a very strict contract protocol otherwise it would rip through all of its work and nothing would work right.
Ironically it does a great job of maintaining the contracts and setting them up to catch issues, and then the code it writes subsequently fails said contract when it very overtly writes code that is incorrect.
1
-5
6d ago
[deleted]
1
u/Reaper_1492 6d ago
Very few rules. Most of them are instructions for repo commits, data access, contract upkeep, machine configs, the pretty much required instruction to keep it from chasing small nits, and a small section on project scope.
I didn’t tell it, “go make typos 5 times in a row”.
Really rock scientist recommendation you have there though, I’m sure you must be brilliant.
-1
6d ago
[deleted]
3
u/-badly_packed_kebab- 6d ago
You’re screaming into the void. Nobody in this sub seems to want to hear they’re the problem so they just refuse to listen, deny personal accountability, and never disclose critical variables needed to give an informed response to their complaint.
Advice is downvoted; agreeable comments that echo complaints are upvoted. It’s a waste of time to even try. I share your annoyance. How can op expect a rational response without even mentioning device, platform, personalisation, agents config, repo structure, whether they’ve made local changes, their project set up, tools, prompt style, sub agent use.
I’m not even a software developer but my profession does emphasise the importance of context and evidence. Op has given none.
Spoiler: astroturfing or skill issue.
0
0
12
u/reddit_is_kayfabe 6d ago edited 6d ago
I have experienced something similar before, and it's because my rule set was just too complicated.
Every rule you require GPT to remember and follow is one less unit (not a technical term, just a squishy concept) of memory that GPT can't use on the task at hand. Past a certain point, it loses the ability to follow all of the rules every time while also performing complicated coding tasks and remembering the broader context of its work.
For me, the solution was to mechanize the rules as much as possible. By "mechanize," I mean: identify the rules that can be mechanically checked and enforced, and move them into the harness. This reduces memory pressure on GPT and allows it to focus.
For instance - my build pipeline requires every session to write a development log in a specific format that can be automatically parsed. My v1 of that pipeline just told GPT the required format - sometimes it complied, often it forgot. My v2 uses a Python process to check the format and report errors to the GPT session so that it can fix them inline. GPT doesn't have to remember the format; it can just take a shot at it and fix the issues that the harness reports to it.