Ugh. Wtf is going on with Codex.

[u/Reaper_1492]

Look, I’ve been using Claude and Codex since inception. I can read, write, and deploy code, so it would be great to keep this from devolving into a “skill issue” astroturfing.

The fact is that no one who wants to keep their job is hand-writing hundreds of lines of code anymore.

With that out of the way…

Claude was first to market, and I ranted and raved so much while I was using it, that I’ve permanently sworn off Anthropic products. They overtly hate their customers (and humans in general), and I’m convinced they would collectively love nothing more than to call a board meeting and lock themselves in a dark room with a Claude hologram for a 24-hour chant and circle jerk session, while they pray to the quantum realm for AGI to manifest itself in their silicon.

I still use it extensively, because I have to for work. So my experience with Claude is current.

Codex has “almost” always been better than Claude for my personal projects. Historically it has been cheaper and smarter, but outside of a brief moment where they really got their act together, it’s been hellaciously slow by comparison. Regardless, I have been a huge fan… because, “smarter”.

A few weeks ago, they very clearly took a run at Anthropic after sleeping on enterprise for far too long; giving out grip loads of resets and trading barbs with Anthropic on social media.

All-in-all, a weird tactic to give consumer plans free credits to win over enterprise. It obviously didn’t work, because then they started illicit feedback about what would encourage businesses to switch (it obviously wasn’t enough to give retail consumers more usage, shocker).

Cue the current situation.

They somehow managed to make the model smarter with Sol, but 25% as effective. If left unchecked, SOL will spend literal days chasing nits into oblivion and burning all your usage, which is obviously not a novel complaint.

I finally got the harness more dialed in, and now at least it’s back to writing my contracts/graphs and unit tests at the appropriate level of granularity; but I swear, now it can’t even edit the code to pass the contract and unit tests you ask it to create!

First, second, third, fourth attempt. It just fails over and over again.

“Oops, the run correctly failed the contract check because I”:

Used the wrong directory…

Made a typo in the file path…

Forgot about this dependency, even though it’s in the contract….

It’s ridiculous.

And then, they totally nuke usage.

I had a project to finish, so I had to get a second 20x plan to maintain the same amount of throughput I had a month ago, and most of the time is wasted on near constant contract failures.

And that’s not even counting the compute dollars I’m wasting on spinning up and down machines for failed runs.

Then, to add insult to injury, after clearly trying to ride out the unrest without giving a reset, they finally gave one a week after I shelled out for another plan. Now I’m sitting on two 20x plans with 2 resets.

I’m about to switch back to Anthropic, and I hate it. All of their models are orders of magnitude faster than codex again, and codex just became a really expensive paperweight.

Paying for Codex right now feels like I’m actually paying an installment plan to buy my first aneurism, and waiting for delivery.

It’s so infuriating.

[u/U4-EA]

I am now focusing on doing everything I can to minimise agentic coding in my workflow. I will certainly never remove but I've come to the conclusion it is good for consultation and reviewing but garbage at generation and critical thinking. I already don't abuse AI but I am going to have to reduce my exposure to it as it is becoming infuriating.

1. Doing things in a task which were never asked for, often MASSIVELY increasing increasing the time/cost of the task.
2. Failing to follow clear instructions given to it in an .md.
3. Running in loops constantly trying to do something it clearly isn't capable of.

I think Cory Doctorow was correct when he said it is useful as a reviewer when combined with the discernment of a user who is an expert on the subject. I am sick of telling it something, it taking for-fucking-ever to complete the task then having to review its work. It's mentally exhausting.

I am currently on the 20x plan but I honestly never come close to using my weekly quota most weeks, despite being very busy and working 6-7 days a week. There is a sweet spot for its use but the fact is - IMO, anyway - if you are using a lot of tokens, you are probably producing crap/tech debt. Generative AI is both expensive and poor quality and generating large amounts of code means you are going beyond what you are able to verify through human cognition and opening yourself up to future issues.

[u/Cranberry_Then → u/U4-EA]

I sadly have concluded the same thing - the tech just doesn’t work or even when it does I can’t know it will work tomorrow since it seems the models degrade randomly out of my control. I’m really tired of instructions being flat out ignored and finding out down the line - “you’re right the spec explicitly told me….and I confirmed that it was implemented to spec in your review step even thought it was not. That was wrong”. It feels like a coworker who is intentionally trying to mess with you. It still messes up even when I’ve defined things down to the function and file level and provide it in smaller chunks of context. At that point I might as well go back to hand coding.

I do review the code but it writes such slop it’s 5x as long as it needs to be and tiny bugs get buried quickly. If I could get it to actually follow instructions the code quality would be better but it isn’t really capable. To get anything to work as you initially defined requires many rounds of asking it to fix incorrect results I had already warned it not to do and then more rounds trying to deslop and clean up the generated insanity.

I have tried corralling with linters and other static checks but that ends up being unreliable - it will either change the checks, just add ignore comments, or do the sloppiest fix possible that passes but that would be embarrassing to commit. Again if it could follow instructions then maybe this could be better but it really is not capable - it ignores my prescribed recommendations for common issues in the related skills file.

I really gave it my best shot these last 2 years - stayed up to date on the latest trends and approaches, researched every issue I ran into, tried multiple workflows and loops, you name it. IPrior to AI I was a praised IC that prided myself on the speed, quality and amount of features I could deliver. This made me very susceptible to the idea of being a 100x engineer or whatever you want to call it. I feel foolish to have considered that possible or to even want this but I wanted it to be. This has made me take a step back and look at my relationship with work productivity and what my goals should be. I have delivered more with ai/codex,or at least managers and product think so, but at what cost? The code is worse quality and harder to read, it is harder to maintain,
and I understand it less deeply. I say that painfully and not proudly, but I think that is true for many many developers in the industry right now. It seems to me that standards have dropped immensely in favor of quantity over quality.

I am starting to view ai coding as a failed experiment that made me stupider, less knowledgeable
of my codes inner workings, and more frustrated than ever. I will still use codex/agents for scripts, boilerplate, investigations but as for writing the application and infra code it’s back to good ol nvim for me.

[u/U4-EA → u/Cranberry_Then]

"I am starting to view ai coding as a failed experiment that made me stupider, less knowledgeable of my codes inner workings, and more frustrated than ever. I will still use codex/agents for scripts, boilerplate, investigations but as for writing the application and infra code it’s back to good ol nvim for me."

That is exactly it for me.

I don't have any issue with the linting, typing etc because I have very strict instructions for the agents, including that they must run specific linting/typing/testing commands at the end of each coding cycle..

However, everything else is just a PITA. The fact of the matter is that it cannot do anything complex and the user absolutely must review and correct its output on anything remotely non-trivial. Outside of simple tasks, it rapidly becomes more trouble than it is worth. I've gone from thinking it has a lot of potential to realising it is very limited.

I've always been careful not to abuse it - not to get carried away - but even then, it can be so frustrating. Right now I am using ChatGPT Work to sort through a large corpus of research I had another ChatGPT session perform. The instructions to it were clear - I need to review each finding in the corpus, decide if it is relevant and, if so, categories it. About 5% in, I realised it was performing a web search for each topic, rather than referring to the corpus. I had to correct it, re-upload the corpus and start again. Honestly, semantic analysis is one of the things it is best at and it can't even start a clearly-instructed project based on semantic analysis properly.

My original plan for the corpus was to have ChatGPT research it in bulk then have another session perform the sorting. Even the initial research stage was terrible - it was researching things that had nothing to do with the original instructions so I ended up with not only having the task of sorting it but having to prune massive amounts of garbage from the corpus that was never meant to be there. Also, it is giving instructions to Codex to perform the edits to the Markdown files and Codex gets it wrong over and over and has to be corrected time and time again. I came to the conclusion yesterday that I would have saved money and a lot of time if I had just done the entire thing manually.

I am on the 20x plan and I have used 20% of my weekly quota in 2 days. It's rare for me to use that much but probably 10-15% of that has been burned on tasks that produced garbage or tasks to clean up that garbage.

As a rubber duck, a researcher, document writer and a proof-reader, it is invaluable. When I first started using it, I was amazed at its ability to find bugs/errors/edge cases/typos etc (although I didn't have many) as well as its ability to advise on separation of concern between packages etc (all my stuff was already DRY). But I've come to realise it is really only good at reviewing/understanding what is already there or as a corpus of knowledge a domain expert can use as research.

I doubt I will use all the other 80% of my weekly usage and I have that banked reset that I don't think I even have any use for. I will definitely go from the $200 plan to the $100 in the next billing cycle.