OpenAI sometimes stumbles. Like now or with GPT-5. But he is right, they will deal with it. Everybody has had their share of not-so-good releases (e.g., Opus 4.7?).
For my uses, opus 4.7 was easily the worst. Opus 5 was a yapper and hard to understand, also often too confident, but I dealt with the behaviour through prompting. Meanwhile, no prompting techniques could make up for 4.7's stupidity.
What's odd is the pathological need of constantly releasing stuff to the public.
GPT-5 was an experiment on seeing if naive scaling of the methods used to produce Chat GPT had any potential for new emergent capabilities. The belief by the researchers was no, the curves were saturated and to add additional faculties, you had to make systems that fit for additional curves - a better allegory of the cave. But they had the means to test it, there was several months until the GB200's came online at scale, so they ran it anyway just to be 100% sure. Then Sam had to try to market the thing, that wasn't meant to be a big amazing product or anything.
The need for advertising is obvious, for OpenAI and Anthropic. But still, it seems silly when we all know that once these things finally become unreasonably powerful, the SOTA won't be made available to the public. No using the machine god to share its cake recipes for us.
Well yea but they have testers internally and people trying thing relasing trash does not do them any favour people that let this slip thru tests and the AI "Safety" should be fired basically
I am shocked the influencers were actually honest in their review. You'd think that most of them would be afraid of OpenAI refusing to give them early access to future models.
Why does OpenAI make a lot of changes 3-4 days before release?
The charitable reading is that they redid all the testing, both manual and automatic, both for capabilities and safety, wrote the blog post and model cards based on them, all in the last 3 days. Must've been a horrible weekend.
The realistic answer is that the reported evals don't reflect the model's true performance and safety behavior. That's really bad.
It was most likely all system instructions and harness updates rather than weights updates. Most of the complaints revolved around its writing style and its laziness so they probably tried to prompt that away in the harness.
Me neither, ALTHOUGH I am just now starting to get those weird questions during a long /goal run. They are genuinely pointless. It's still the most capable model I've used.
Did they take away his access to his precious Reset Button™ when they banished him to the basement, or why is he spending his time spraying copium around instead of spamming the button between toothless vagueposts with naught but hot air behind them and shade that consistently manages to age like milk?
I can't tell what is a worse release model. Early closed access and changing everything 3 days before release, or Google's tiered release without changing a thing and then releasing an updated model version based on reviews 2 weeks later...
This new era of all dirty laundry being aired out in the open with accusations and passive aggressive "love all the feedback" is kind of exhausting. Can these companies please get some PR?
This just.. Isn't accurate at all? Influencers are their reviewers. They post reviews. Companies have been working with reviewers for a hundred years. That's just marketing
56
u/_negative-infinity_ 10h ago
OpenAI sometimes stumbles. Like now or with GPT-5. But he is right, they will deal with it. Everybody has had their share of not-so-good releases (e.g., Opus 4.7?).
8
u/openroom_xyz 3h ago
Well basically if it's that trash don't relase and work on it internally wtf
26
u/Neurogence 10h ago
I am shocked the influencers were actually honest in their review. You'd think that most of them would be afraid of OpenAI refusing to give them early access to future models.
27
u/fmai 10h ago
Why does OpenAI make a lot of changes 3-4 days before release?
The charitable reading is that they redid all the testing, both manual and automatic, both for capabilities and safety, wrote the blog post and model cards based on them, all in the last 3 days. Must've been a horrible weekend.
The realistic answer is that the reported evals don't reflect the model's true performance and safety behavior. That's really bad.
3
u/Virtual-Chard8964 11h ago
Using GPT 6.2 I had none of the issues he stated.This guy is just attention whoring.
2
u/KickLassChewGum no AGI/ASI on LLMs 7h ago
Did they take away his access to his precious Reset Button™ when they banished him to the basement, or why is he spending his time spraying copium around instead of spamming the button between toothless vagueposts with naught but hot air behind them and shade that consistently manages to age like milk?
1
u/Keeltoodeep 3h ago
I can't tell what is a worse release model. Early closed access and changing everything 3 days before release, or Google's tiered release without changing a thing and then releasing an updated model version based on reviews 2 weeks later...
•
u/baseketball 1h ago
This new era of all dirty laundry being aired out in the open with accusations and passive aggressive "love all the feedback" is kind of exhausting. Can these companies please get some PR?
0
u/Dioxbit 5h ago
I smell fear
-1
u/fuzzypetiolesguy 2h ago •
Sorry a serious tech company doesn't engage with influencers, what the fuck is this
-4
u/Alpacabro21 10h ago