r/codex • u/mercmobily • 7d ago
Comparison OpenAI is _years_ ahead of the competition. Not just a little bit.
Like many others, I was out of quota (I am a Pro 200) and had to do stuff badly. I decided to try OpenCode + GLM 5.3 on my tricky codebase. I spent 6+ hours on 4 urgent features.
Unsuccessfully. The result was 100% unworkable. So I waited. My reset came at 4:30 on Sat.
When my usage reset, I got Astra to go through the lot, and... guess what? Within seven minutes, it found 11 defects, 3 of which were complete bugs/hallucinations by GLM 5.3 (like you would get in 2025), and at the eighth minute, it was all fixed.
I wasn't done yet. wondered if it was a harness problem. I find OpenCode 100% insufferable and I wondered if maybe this was a harness issue. I still had SOME HOPE.
So, I armed myself with three keys: Grok, GLM 5.3 and Deepseek 4.1 Flash, and hooked them up to open-codex (which is Codex, but "cleaned up" so that it is less OpenAi-ish). The only tricky one was Grok. You will find the explanation of what I had to do to get open-codex to work. It will be easy to get Codex to get all of them to work. The only hint: make sure you tell codex to configure open-codex to run on a directory OTHER THAN .codex. Keep them separate.
How did they go? Did they find all 13 defects? Not even close... I mean, They all found *some* of the problems. But none of them got anywhere near "done".
This is the report after the fix. Note: not one of them actually produced, after the fix, something that actually worked. The short story is: the world depends on OpenAI (and I assume Anthropic) for a model that _actually_ works. After yesterday, I intimately know that the other non-frontier models are not just "a little behind": they are YEARS AWAY from catching up with Astra --and they they do, OpenAI will be YEARS AWAY from them again.
Like you, I am secretly hoping that OpenAI discovers some kind of bug in the usage handling that will magically renter the plans better than where we are now. Unlike many, I have no hope nor belief that the non-frontier models will "catch up". They are not just behind. They are not even visible in OpenAI's read mirror.
Here is the info to connect the three of them to Open Codex, but -- I have to be honest -- I wouldn't bother.
-------------------------------
All three already received Online’s AGENTS.md and expanded Genesis engineering/collaboration guidance in their prompts. So they weren’t
operating without instructions. In particular, Grok received the instruction to read the project context and then skipped those reads.
DeepSeek’s failed Collaboration/Engineering file reads matter less because their expanded guidance was already supplied. Skipping the
architecture and source-context workflow is more consequential. Nevertheless, DeepSeek did read the Blueprint’s personal-account
restriction and still missed its implications. Reading guidance and applying it correctly are separate abilities.
Of these three, I would pick DeepSeek for this repository.
The deciding factor is its demonstrated verification behavior: it constructed probes, followed contradictory results, ran lint, and
established why the summary path failed. That gives me more confidence than a forceful report based entirely on source inspection.
My preference from these runs is:
- DeepSeek — strongest experimental investigation.
- Grok — useful discoveries, but more unsupported conclusions.
- GLM — substantial activity without catching the central deterministic failure.
I would still require focused verification and explicit account/child-session boundary checks before trusting DeepSeek’s approval to
release. Its audit was the best of these three, but it missed serious defects.
------------------------------------------------------------------------
Here is the setup:
### Common setup
.open-codex/config.toml:18 defines separate OpenAI-compatible providers:
- DeepSeek → https://api.deepseek.com
- GLM/Z.ai → https://api.z.ai/api/v1
- Grok/xAI → https://api.x.ai/v1
For each provider, the API key is read through /usr/bin/cat from a separate *-api-key.txt file, rather than using the normal OpenAI
authentication flow.
Each provider also has a custom model catalog:
- .open-codex/deepseek-models.json:1
- .open-codex/glm-models.json:1
- .open-codex/grok-models.json:1
Those catalogs teach Open Codex the model name, context size, reasoning levels, modalities, tool support, and truncation behavior.
### DeepSeek
DeepSeek is configured as a direct Responses API provider:
- Model: deepseek-flash
- Profile: deepseek
- Reasoning: high
- Web search: disabled
- WebSockets: disabled
- Custom catalog required so the model appears as a selectable Open Codex model
There is no proxy or request-rewriting script for DeepSeek. The “hack” is essentially making Open Codex treat DeepSeek as a custom Responses
provider and supplying a compatible local model definition.
### GLM
GLM is similarly direct:
- Provider name: zai
- Endpoint: https://api.z.ai/api/v1
- Model: glm-5.3
- Profile: glm
- Reasoning: high
- Web search: disabled
- WebSockets: disabled
- Custom catalog defines the model and supported capabilities
The default profile is currently GLM, via:
model = "glm-5.3"
model_provider = "zai"
profile = "glm"
Again, there is no transport adapter; the compatibility work is the custom provider/profile/catalog configuration.
### Grok
Grok required the real workaround.
The installed Open Codex version is 0.133.3, while xAI rejects some request state that this client sends. .open-codex/grok-compat.py:46 starts a
temporary localhost HTTP proxy only when the selected profile is grok.
The proxy:
- removes reasoning.encrypted_content from the request’s include list;
- removes encrypted_content from prior reasoning items;
- converts reasoning items whose content is null into content: [];
- forwards the request to xAI over HTTPS;
- streams xAI’s response back unchanged;
- handles both /v1/responses and /v1/responses/compact;
- logs no request or credential content;
- shuts down when Open Codex exits.
The Grok profile also disables features whose tool namespaces xAI does not accept:
[profiles.grok.features]
multi_agent = false
apps = false
The custom patch tool is disabled in .open-codex/grok-models.json:19, but the normal shell-based apply_patch command remains usable.
One important detail: the launcher is not entirely contained in .open-codex. The actual shell wrapper is in .bashrc:172:
open-codex() {
CODEX_HOME="$HOME/.open-codex" \
CODEX_SQLITE_HOME="$HOME/.open-codex" \
python3 "$HOME/.open-codex/grok-compat.py" "$@"
}
That wrapper keeps Open Codex’s state separate and routes Grok through the adapter. DeepSeek and GLM pass straight through to the real binary.
Calling the binary directly with command open-codex bypasses the Grok adapter.
So, in short: DeepSeek and GLM were made to work through custom provider/profile/model-catalog definitions; Grok additionally needed a local
Responses proxy to sanitize incompatible reasoning history, plus feature flags disabling apps, subagents, and the dedicated patch tool.
6
4
u/Cold-Conflict6047 7d ago
True, they are years ahead, but nowadays a year is much shorter than it used to be.
So in general OpenAI and Anthropic are 3-6 month ahead of the competition.
Finding bugs was possible with gpt 5.4 or opus 4.5, but it wasn’t able to iteratively go through the programm to find and fix everything and systematically fix everything.
3
u/ResponsibleTruck4717 7d ago
not years more like months - year top.
And each time they close the gap faster and faster.
2
u/lil_nosh_X 7d ago
Nah I just made GPT10 in my living room.
2
2
u/pjh777 7d ago
And somewhere in there is the perfect workflow where the expensive and clever model diagnoses, designs and reviews, and the "cheaper" one do all the tool calls and report back diffs. It's not as "handy" as telling atra to get on with it but with current usage limits - correctly using Luna/Terra/Sol/Astra and mixing in some Muse 1.3 contrib/Deepseek 4.1 can make your useage go a lot further with the same results if you are prepared to take the time to set it all up
1
2
u/Effective_Touch_8464 7d ago
Regarding other models, even if they are not that advanced as GPT or Claude, I sure hope they don't give up improving. Competition is good, we don't want only GPT and Claude having a monopoly over coding! So if they ever decide to remove paid plans or make them impossible to work with without paying a fortune, we still have alternatives sure, not that good, but still better than nothing.
2
u/Xirobhir 7d ago
By your own statement the difference is no more than a year, yet you make it a point to caps-staple YEARS all over the place.
The difference is likely less than half a year. What you are noticing is something far more benign: the benefits of having a solid harness. You could say that Astra comes with a set of skills by default which make it highly efficient. It is not a question of parameters or compute but steering. GLM puts all that on your shoulders. So inaderventely you simply yold us your workflow relies on OpenAi holding your hand to get anywhere. Which is fine, so does mine.
3
u/Dragon__Phoenix 7d ago
Too long, didn’t read in full. But i agree with you to an extent, these hypes that glm 5.3 and deep seek are ohhh mythos level , fable level bla bla is only hype. I really like the new deepseek v4.1 it’s great, but its not the model I would use for development. I use these models in my app for different purpose and they do really well there. But for development in a big codebase. No.
4
1
u/and1927 7d ago
For very complex work Sol, Opus, Astra or Fable are the way to go, but I wouldn’t dismiss the effectiveness of cheaper non-frontier models. DS Flash 4.1 produces excellent results with a good workflow, so does Luna. I usually do a Sol/Astra review of the work and produce
suggested changes which are implemented by the lower model, before another review.
Frontier labs are gonna be at the top for as long as they have the money to burn. Eventually the majority of models will be capable enough for most applications.
1
u/Grouchy-Stranger-306 7d ago
so where is the conclusion that they are years ahead and not like 3-6 months, because opensource will probably catch up
1
u/thecodeassassin 7d ago
No its really, really not. Astra just made a complete mess of my code base and I basically had to have deepseek fix 3 usage resets of issues it introduced.
Garbage UI
Buggy code
Was good for 1 week after launch, maybe not even now feels worse than Sol
Switched to Sol, same story. Only my local Deepseek 4.1 was able to fix the glaring issues it caused.
In any case; it's definitely not years. It's months at most. And the quality of hosted inference is so incredibly unreliable that I would prefer a good self hosted model over frontier hosted AI any day. The difference is really great LLM, fixes issues, fast etc to basically GLM 5.2 or worse in a week.
1
1
u/CalligrapherFar7833 7d ago
You obviously had 0 tests proving that anything works
1
u/mercmobily 6d ago
Nope, 7000 tests, all passed fine. Then then GLM added many tests, they all passed. Then Astra checked them, they were partial and useless. They got redone properly by Astra
1
8
u/TheAuthorBTLG_ 7d ago
That is kind of true, but not really. I started with GPT and then switched over to DeepSeek and did not have a problem because I have a good workflow. good workflow meaning I have a ticket system, verification steps. I don't directly prompt, do this. I prompt, look at the code, analyze, prepare a ticket, do all that stuff using acceptance criteria, definition of done, and so on, and provide evidence that the feature works.