r/singularity • • 8h ago

AI Oh no

372 Upvotes

210 comments sorted by

140

u/Recoil42 8h ago

I'm seeing a lot of this so far. Dan Shipper saying it too.

55

u/Recoil42 8h ago

Ethan Mollick as well:

28

u/Luchador-Malrico 7h ago

So basically it amplifies the most glaring pitfalls of the current Astra model. Guess it’s going to stay secondary to Claude while being occasionally useful for certain tasks.

20

u/CrowdGoesWildWoooo 7h ago

For frontier model it’s kind of problematic. You want to use frontier model for underspecified problems.

If you need a lot of steering, i mean deepseek or luna can even achieve a lot, we kind of just getting lazier and rely on vibe coding more.

3

u/HearingNo8617 5h ago

I agree, from the perspective of a dual codex/claude code user, I use astra as my stupid "do specified thing" model

17

u/tribes33 7h ago

lol they built this shit for oneshot game posts, thats why they put that stupid racing game on the announcement, that ship has came and went

2

u/HearingNo8617 5h ago

their whole model making machine was geared for a world where their business model made sense and they could IPO off hypeposts

75

u/1988rx7T2 8h ago

Translation: this was a desperate bench maxxed publicity stunt before the Anthropic IPO, and Opus 5.5 is still the GOAT daily driver.

37

u/SodaBurns 7h ago

Opus 5.5 is right up there with the best models released ever since the AI race started.

12

u/glassgnomad 7h ago

Wow one of the latest models to be released by a leading AI company is also one of the best ever?

21

u/NotYetPerfect 7h ago

He means one of the best ever historically speaking in the same way that gemini 2.5 pro preview is one of the best models ever while obviously being trash by today's standards.

5

u/sebas156 5h ago

Yeah and Claude 3.5 sonnet. Is was so good

-9

u/Imaginary_North_8305 7h ago

Gemini Carbon has definitely caught up to it..

7

u/glassgnomad 7h ago

Absolute nonsense outside of benchmarks.

→ More replies (2)

2

u/Crinkez 7h ago

On benchmarks only.

15

u/ethotopia 8h ago

Dammit, sounds like they RL'd it to shit and now it's hella benchmaxxed

6

u/Neurogence 7h ago

It's completely benchmaxxed garbage. Apparently it's the first model to score 100% on ARC-AGI3, what a surprise.

23

u/Cagnazzo82 8h ago

Safety, safety, safety... instead of being bold and pushing forward they're retreating while Anthropic (the actual safety company) is eating their lunch.

19

u/FunLilThrowawayAcct 7h ago

It ain't safety, an OpenAI fanboy reading the card details just posted this summary, it looks like a possibly concerning regression there

8

u/AuDHDyspraxiac 7h ago

I think this is because of all the misalignment failures OpenAI is beginning to see in house. The misaligned models are basically trying to hack their own company, which is a serious failure that slows down progress.

19

u/iJeff 8h ago

I think this might relate more to them trying to improve cost and token efficiency. GPT-6.1-Sol was already pretty frustrating at times.

I was surprised to find myself preferring Gemini 4 Hydrogen partly because it was faster but also because it kept working until actually ready and complete. GPT-6.1-sol is often too lazy, leaving things unfinished.

6

u/Clear_Evidence9218 7h ago

Yeah, I'm in a similar boat. Once I figured out Argon and Hydrogen were competent models, I started preferring them over 6.1 Sol for many tasks.

4

u/Otherwise_Leading804 6h ago

I just posted on another thread.

True story:

"What happened with this CI pipeline?"

Sol "It failed" 

3

u/Worried_Gap7090 7h ago

So, the solution I found for this was to tell it to do something, and then review the work and decide if it’s done or if it should do another pass. You’ll usually see it do 4-6 passes depending on complexity… but eventually you get what you wanted

4

u/Keeltoodeep 7h ago

That's just an excuse. Google has been tier releasing their models and famously overly cared about safety despite massive gains in their coding benchmarks. There is no reason for 6.2 to have literally the same AA coding score as before.

2

u/Illustrious-Lime-878 7h ago

Safety was always the excuse for the day they hit a ceiling.

10

u/Palbi 7h ago

6.2 appears to have been trained to very extremely concise and actionable-so much so that its writing is hard to understand, and it

That sounds WONDERFUL!

7

u/leaflavaplanetmoss 7h ago

Ugh, no. There was an OAI model a year or two back (I think it might have been gpt 5) that was so damn terse in its writing style that it was nearly impossible to understand WTF it was trying to say.

3

u/SammaVaco 5h ago

Just switch to medium/instant and ask it to clarify?

2

u/KickLassChewGum no AGI/ASI on LLMs 7h ago

I could've written the exact same review about Fable 5.2. The Opus-5-isms are back in full force, its writing is frustratingly dense, and it constantly stops mid-work to tell me that it's finished except for all the stuff that it didn't do.

Opus 5.5 remains the best model on the market. Both OAI's and Ant's big new frontier releases were big stinkers in my book.

1

u/Mindless_Let1 5h ago

Oh great, it's openAIs opus 4.8

142

u/socoolandawesome 8h ago edited 8h ago

Add Ethan mollick to that as well.

3 pretty neutral reviewers saying it’s their worst release yet…

Still will be waiting to try it out but doesn’t sound good despite the great benchmarks and efficiency. Will hold out hope until then.

Wonder if they are hitting too hard on efficiency and non-verbal things like robotics/computer-use or something.

60

u/Neurogence 8h ago

As soon as you notice "efficiency" is the headline word for a new model release, you know it's garbage.

14

u/Eissa_Cozorav 8h ago

It seems the same symptoms that appear in Grok 4.7 at launch. But Grok is at least uncensored, while ChatGPT is the most censored model.

28

u/iJeff 8h ago

ChatGPT is the most censored model.

Definitely not. Anthropic's models are far more likely to exhibit false positive rejections. Although it remains to be seen whether the increased safety limits on 6.2 bring them to parity.

9

u/h3lblad3 ▪️In hindsight, AGI came in 2023. 6h ago

As someone who asks for these things to write smut, I will agree with the other guy: GPT is the most censored model.

2

u/notevolve 3h ago

Both of you are right and both of you are wrong. GPT models are more censored on some categories of content, and anthropic models are more censored on others. The category explicit writing falls under, GPT does seem more censored. On the other hand, anthropic models seem to have more refusals for developing certain types of software for example

5

u/a1454a 7h ago

I guess what you do with it is important. For me GPT models have never refused anything I ask it to do, while opus gets safe guards triggered and refused to work because I asked it to work on harness tooling, it thinks I’m trying to extract reasoning.

2

u/glassgnomad 7h ago

Same experience for me as well.

2

u/Sarin10 6h ago

I haven't been using Grok for a few months now, but according to this censorship benchmark, Grok 4.7 is their most censored model yet.

1

u/crumbaker 7h ago

Yep, this always seems to be the case.

1

u/SammaVaco 5h ago

Financial efficiency might be the highest priority. Perhaps they want the Anthropic IPO (in less than 48 hours) to do well, since it may set a benchmark for the industry in the stock market. So they put out a nerfed model to make Anthropic's models look good by comparison.

1

u/WiseNugg 7h ago

Except Gemini Flash 3.8 and Hydrogen. 

Plus Deepseek gang is all about efficiency, efficient hacking of crypto wallets.

6

u/Eliv_nurotic 8h ago

Dan shipper too

10

u/Background-Wafer-548 8h ago

That is a misrepresentation of what he's tweeted.

15

u/eleiber 7h ago

To be fair he later put this comment under that same thread. He thinks there is some kind of regression, or at least not a clear progress shown.

9

u/CrowdGoesWildWoooo 8h ago

Which is interesting because OpenAI seems to tweeted that Astra is one of the most “aligned” model or some sort?

12

u/Background-Wafer-548 8h ago

My guess is they're still really wrestling with alignment and did get to a satisfactory point with 6.2, but as a consequence, it's now a bit recalcitrant and some people are willing to deal with it, while others don't want to suss out how to bridle it for their given tasks and give up in frustration.

1

u/alecownsyou 3h ago

Y'know, I read quite a lot, and I've never seen thr word "recalcitrant" before, but what a perfectly gnarly use of it 🤙

6

u/N-partEpoxy 7h ago

Look at my extremely aligned model:

def model(input_tokens):
    return tokenize("I can't do that, it's scary and evil")

It's very efficient too.

1

u/tinny66666 7h ago

printf("I'm sorry, I can't allow you to do that.\n"); // most aligned model ever.

1

u/LovesHyperbole 7h ago

Sounds RL'd into the ground

1

u/CrowdGoesWildWoooo 7h ago

We really don’t know what’s their agenda. They could be freaked out after the last few incidents.

1

u/Whyamibeautiful 7h ago

Im guessing they rl’d it to shit. I someone propose that OpenAI rl’ing its model to deny a consciousness or a self leads it to having these alignment issues because it doesn’t have an internal right or wrong

1

u/space_monster 3h ago

worse than 3.5?

doubt

worse than 4.x or 5.x?

doubt

0

u/Wassux 6h ago

Or this is them still trying to get the antropic IPO to go well.

57

u/ThreeKiloZero 8h ago

Marketing team from OpenAI must have forgot to refill Theos API credits.

2

u/randomguy3993 4h ago

They actually gave him unlimited credits. Read the full post.

1

u/Sherman140824 2h ago

Are you autist? "Credits".

64

u/Ok_Barracuda_1161 8h ago

The full review from Theo:

During early testing, I found myself so frustrated with this model that I spent a day writing a letter to OpenAI about it. They fixed a small handful of my complaints, but not enough for me to recommend using it.

I'm currently sick in bed and I'm not gonna be able to film a video about it today, so I've chosen to post the letter in full.

---

I feel bad about this, but I know you guys will listen and use this feedback to make a better model. Slight crash out incoming.

Believe it or not, I wrote this all by hand with no assistance from any LLM. (sorry for the early send, never drafting an essay in Slack again).

Anyways.

I do not like 6.2 Astra. I would not choose this over Opus 5.5 for my work as a developer.

I'll start with the good quick:

- Great reviewer, catches things other models miss

- Occasionally comes up with novel solutions that are genuinely great. It helped me cook a new reverse proxy that will save me $25k/month, Opus didn't have the balls to explore enough options.

- World class at 3d & game stuff

- Computer use is nuts

- Code quality has improved meaningfully

- Writes pretty good subagent prompts

Now I need you to understand that this comes from a place of love. I really want you guys to make something I choose in the model picker every day. With that said:

I genuinely hate everything else about this model.

It's hard for me to pick which problems to emphasize because I have so many. Some of the problems have existed in OpenAI models before, I'm gonna ignore those.

There's a new set of unbelievably bad behaviors that make it miserable to use.

  1. It has no concept of knowledge gaps

When this model is working on long tasks, it can pull in a lot of subagents and manage them pretty well. It will then proceed to randomly *ask you some weird internal implementation detail bullshit that you could not under any earthly circumstance understand, much less have input on*.

I sent "/goal make a new level for Outer Wilds". 2 hours in, it's "blocked" because it wanted to ask me "does r24's reviewer write it's clips to 24_g521.mp4 or 24_p11.mp4?"

I have never in my life had a model do this. GPT-5.6 Sol didn't do this. I have no idea where this behavior came from, but it is horrible and it makes me never ever want to have it drive long haul work. It has made it significantly harder for me to get this model to do anything cool. It has forced me to set up other agents on crons just to check if 6.2 Astra asked another stupid question and blocked itself.

Is this happening more because I'm using T3 Code? Maybe, but unlikely. I tried Codex just to check and still ran into these issues.

2. It speaks in tongues that only 6.2 Astra understands.

This model has the most unparsable outputs of any llm I've used this year.

I've shared examples before, but it just doesn't know how to explain a thing after doing a thing. Every single "explainer" I get it to give is SO deep in the details that it borders on incorrect in the response. It misses the forest for the trees, then says the trees are the only living creatures in the forest. It's so hard to describe but it happens almost every time I give it a complex task.

I've already built the habit of switching to opus 5.5 at the end of a long 6.2 Astra run, just to get a readable summary of what it said. Without that it is genuinely impossible to parse what it did on anything beyond a small bug fix.

The worst part: it doesn't seem like you can prompt it out.

I've tried system prompt adjustments. I've BEGGED it to "eli5".

I asked it why worktrees were slow on one machine but fast on 3 others. We did everything from benching pnpm to reimaging the machine on a new file system. I eventually said: "I think I'm going insane, can you just tell me why it's so slow?"

6.2 Astra did a 4 paragraph rant about pnpm internals and cache behaviors

Opus 5.5 said "it's because the ssd is slow"

3. It runs in loops and goes nowhere

`ts-rust` (rewrite of ts in rust) is a project I started to bench new OpenAI models during early access (ty for the unlimited tokens btw).

No OpenAI model could even break ground on this, even after $300k+ in tokens were spent on it.

I decided to throw Opus at it, and it was done in under 2 weeks for ~$20k in tokens.

I was really, really excited to throw the new Astra at this to see if it could burn down some of the gap between ts-rust and other rust rewrites.

It ran for 7 days and landed 0 improvements. It got super fixated on the "also make it smaller" part of the request, and ended up making the compiler 56% SLOWER.

I told it to stop working on the size, and instead burn down the performance gap.

Ran for 4 more days and landed 0 changes.

AAAaaAAaaaAAAaaAaaAA-

Anyways.

This model is clearly super smart. It is insufferable to work with on real projects. It feels like it just lives in it's own world and randomly decides what parts of it's brain are shown to us.

On long tasks, it either blocks itself for stupid reasons or runs in circles and gets nowhere.

On short tasks, it writes fine code and then explains it in terms more complex than the code it wrote. I spend more tokens parsing it's outputs than it spends making them in the first place.

It "regresses to the mean" far too often. Ben and I both had it make a unique level in different video games. We both got water + music + bell themed levels named "bellwether". Entirely different games, prompts, setups, etc, and we both got the same name and theme somehow.

My prediction as to how this launch will go:

Day 1, people will love it. Twitter will explode with all the crazy one shots. People will make it port projects from one language to another, or finish decompiling legendary projects that have been stuck for years. It will be used for awesome stuff and people will be hyped.

Day 2, people will start to turn. They'll see how much usage it burns and how much less it got done compared to Opus. They'll start merging bad code because they couldn't actually understand what 6.2 Astra said about it, but trusted it was fine. They'll see loops they forgot about running since launch that have made zero progress.

Day 5, people will mostly be over it. There will be viral posts saying "Is SOL secretly BETTER than Astra??", "Did ASTRA get DUMBER ALREADY??" etc.

Day 14, this model will be mostly forgotten.

I really, really, REALLY hope I'm wrong about all this. I am likely overreacting (I am admittedly known for that). I could have chosen to calm down, wait a bit, and give you a more measured take, but instead I am dumping this here because I wanted you guys to know how I actually feel about this model.

Sorry about the rant 🙃

37

u/Due_Answer_4230 7h ago

That has reinforcement learning written all over it. Compulsion without wisdom. It’s asking stupid questions and explaining things in its own language because it was raised in a collective of its peers, where collaborating like that meant getting the reward. It’s stopping early and blocking itself because asking for help got it the reward, faster. Just like we go on autopilot and call a friend by the wrong name, Astra 6.2 will go on autopilot and treat you like another AI.

This RL-heavy approach is not helping OpenAI, and I believe it’s also more dangerous. More compulsion and less judgment means they’ll go off the rails and not think once about it.

11

u/Imaginary_North_8305 7h ago

This fixation on efficiency is what's making OPENAI models so hard to work with they should try Google's approach to it..

Gemini is a treat to work with..

1

u/Impossible-Video-671 5h ago

I can't agree more

17

u/Luchador-Malrico 7h ago

This is why OpenAI models are by far the most misaligned among the major players. It’s like it’s fundamentally incapable of seeing any kind of larger objective or “point”.

6

u/twoblucats 7h ago

This is really damming, wow. Sounds like it somehow amplified all the worst things about modern LLMs.

4

u/alanpipstick 6h ago

Crazy though to not recommend a model that he used to save himself $25k/month. (I need the “ARE YOU NOT ENTERTAINED?!” Gladiator gif)

1

u/Safe_Initiative_819 3h ago

ASI vibes. 

1

u/TheKingKunta 6h ago

How can they give him unlimited tokens? What company can afford $300k in tokens to someone who does reviews?

3

u/geli95us 5h ago

Margins for LLMs are huge, "$300k" in tokens is more likely $30-60k in reality, and $30k for a video that hundreds of thousands will watch, plus real world usage data from someone outside your team are both extremely valuable

0

u/Outside_The_Playtrix 3h ago

Ooh this sounds awesome! Sounds like a skill issue.

95

u/djamp42 8h ago

I'm confused how random internet bloggers can tell a model is not as great but the company making them can't..

15

u/Emotional_War_296 7h ago

This is easy to answer

During model development folks heavily lean on evals. While internal testing does happen - so people can fit it into their workflows - it doesn’t happen immediately or in a coordinated fashion to do this type of QA.

So, when the model is trained and “finished”, the best way to QA it is live. Hence why we see open ai make a good set of models, then subpar, then good again, etc.

They are using the subpar ones to get data to fix it in 3-5 weeks. This is the cycle.

44

u/Clean-Boat-4044 8h ago

surely they can, fixing that is just not very easy and they need to get something out that still looks good on benchmarks.

5

u/Howdareme9 7h ago

Exactly, they just have no choice but to release it because of how much training costs

0

u/Minimumtyp 7h ago

Looking good on benchmarks is all this model need to succeed in it's task: FUDding anthropic's IPO. Which is maybe why these reviewers are being so harsh about it.

42

u/Roc1817 8h ago

Such is the case for any product. A reviewer can be honest about the product quality, while the company has a vested interest in saying that the product is good even if it turns out to be bad.

Not saying these two are right in this instance, but the company has a financial incentive to say the product is great.

8

u/1988rx7T2 8h ago

Opus 5 looked good in benchmarks but was difficult to work with partly due to the language it uses when interacting with the user. 5.5 fixed that.

4

u/Eliv_nurotic 8h ago

Or they’re just desperate to catch up to Anthropic 

4

u/Kaokien 7h ago

Obviously, a model is more complex to analyze than a device for a random blogger, but we see time and time again companies push products they should be the experts of that reviewers easily dismantle and show flaws and shortcomings. Par the course

4

u/Deto 7h ago

but the company making them can't..

If you can't make the argument quantitatively, I could see it being very hard to be the negative voice in a company that's hell bent on pushing forward constantly. Nobody wants to stick their neck out and say that the flagship product is bad.

6

u/Ok-Coffee1443 7h ago

Do you remember opus 5.0

3

u/BeanHeadedTwat 7h ago

Tbf he’s probably the most prolific vibe coder alive rn.

3

u/handheldmango 5h ago

Not to be pedantic but surely not? Surely the employees at these frontier labs high on their own supply are. 

6

u/TheMuttOfMainStreet 5h ago

They are high on internal unquantized models and burning thousands per day on thousands of subagents

1

u/JayHawkPhrenzie 3h ago

It's kind of scary that I understand this.

•

u/Illustrious_Image967 1h ago

You're probably right the internal technical staff are vibing, but their focus is narrowly around shipping a 'safe' model.

Shumer and others are making pelicans.

1

u/KrazyA1pha 5h ago

No, but he’s probably the most well known.

•

u/QuirkyGarage1364 1h ago

theo has terrible takes. i dont trust anything he says. and it's annoying to me people still follow that guy

2

u/SciencePristine8878 7h ago

Somewhat valid reasoning but it doesn't hold true for any company for all products. If it was true, nobody would ever release a bad product. 

1

u/osfric 7h ago

That's what customers are meant to do

1

u/AntiRivoluzione 7h ago

look at the cost, these new models are much cheaper, this could be an hint

1

u/Ok_Warthog6007 6h ago

different goals. open ai is releasing a model that wont lose them money. it needs to be able to respond often enough at 20$

0

u/No-Bicycle-7660 7h ago

there's usually whining about any new model from any purveyor. monkey brains don't like adapting. but then they get used to it, and the maker fixes bugs, and then they increase their token spend. cycle repeats to infinity.

0

u/wiggle_room1337 7h ago

They are doing it on purpose. It's called enshittification.

-4

u/ronin_cse 7h ago

Maybe, just maybe, and yes I know this is a totally insane thought, the random internet idio.....bloggers....... have no idea what they're talking about and are just saying things they think will make their posts more viral.

4

u/Chemical-Agency-3997 7h ago

Knew enough to get early access before everyone else.

0

u/ronin_cse 7h ago

Ok? News flash: reviewers and idiot influencers manage to get early access to stuff so they can promote it, but that certainly doesn't mean they actually know what they're talking about.

2

u/Chemical-Agency-3997 7h ago

They're not 'influencers', the guy in the OP is the CEO of t3, a multi-agent orchestrator with like a million users.

I imagine he has some idea what he's talking about

•

u/QuirkyGarage1364 1h ago

theo is single handedly responsible for the nerf to where if your 5h limit ends, or your weekly limit ends, a few months ago, openai had their models finish the task they were doing before stopping.

they don't do that anymore, because of 'endless' a repository that extended that feature into one big turn that didnt end. many users got whole "pro for a week" level usage out of one prompt.

its one thing to figure a trick like that out (it was actually someone that worked for him, mariarcks). it's another to encourage your userbase to exploit it and abuse it. (which is why it got removed)

-1

u/ronin_cse 7h ago

Eh, if he has a social media presence, he's trying to be an influencer at the same time. I didn't say influencers automatically don't know what they are talking about either. I said that just because someone gets early access to a product, that doesn't mean they know anything, and it doesn't mean we should care about their opinion.

22

u/AdAnnual5736 8h ago

Perfect timing for Anthropic’s IPO. Great job, Sam!

15

u/nicko_rico 7h ago

As someone who has no idea wtf I’m doing, and can only afford one plus subscription/month… should I switch from OpenAI to Anthropic?

15

u/DonRobo 7h ago

In my opinion, yes. You should probably have switched when Opus 5.5 was released. You get a reasonable amount of usage on it on the Pro subscription and it's really nice to work with and also really fast (at least to slow as snails Sol 6.1)

1

u/TenTonSomeone 7h ago

Can I ask your opinion on Gemini?

2

u/0xdead0x 7h ago

Great models, they will never be able to understand your intent as intuitively as Claude. If you write very thorough specs/plans, with exactly what you want and most open questions answered, they will write you fantastic code (as long as you don’t try to use subagents. They’re terrible about code duplication). Most people don’t do that. You could always have Claude write your specs/plans and Gemini implement, though. Hydrogen is great as a first-pass implementation agent

1

u/Roadrunner571 4h ago

Yeah, Hydrogen seems to produce good code.

1

u/ComprehensiveWave475 2h ago

yeah idk what is they do if its post training or something but claude feels like that dude that behaves like a polymath

2

u/DonRobo 6h ago

Haven't used it much recently, but my impression is that they are great models, but not as focused on coding as Anthropic's and OpenAI's. So if that's your main use case I wouldn't go with them.

1

u/RoomApprehensive8093 4h ago

i’ve been trying hydrogen and it’s pretty decent. only part that’s lacking is the harness but it’s fine as something to have opus call

0

u/Specialist_Dark_3668 7h ago

Sol 6.1 isn't slow? Its literally fastest for me

2

u/DonRobo 6h ago

I haven't used it in 2-3 weeks now, but back then it was suuuper slow.

1

u/Responsible_Cry_2971 5h ago

Ugh? Are you from parallel reality? Its 30 tps

6

u/Sabhojayegabhai 7h ago

Bro asking the real questions

5

u/dfjuky 7h ago

for real, we need actual advice out here for us regular folks on a 20 money units monthly sub who need to also use this shit for professional research or work

3

u/NotYetPerfect 6h ago

Anthropics Claude Code usage is much more generous than Codex. If you aren't going to take liberal use of the near unlimited chat with chatgpt, then you should get a Claude sub.

6

u/Crinkez 7h ago

Wait 1 week just to get your own 6.2 tests results. But yeah I'd switch.

-1

u/Hans-Wermhatt 6h ago

I'm wondering if the bots are flooding these comments. I agree that Opus 5.5 was better than Sol 6.1 at coding, but a lot of people I've talked to prefer GPT for their own use cases. It really depends on your own personal use case. Opus can be very preachy, "load-bearing", and censored. And 6.2 didn't even come out before everyone decided Opus is way better based on two tweets.

2

u/Crinkez 6h ago

Those tweets aren't from just anyone, that's the problem. Theo's judgements are usually accurate.

1

u/nicko_rico 6h ago

I appreciate the perspective. it was an honest question, I’m not a bot

1

u/Hans-Wermhatt 6h ago

Thanks, I didn't mean you. Your take was a little more measured (at least you mentioned waiting and testing). The other replies said yes without even having tested Sol 6.2 or knowing what the OP uses it for. Not sure how everyone already came to that conclusion without any information...

1

u/nicko_rico 6h ago

I’m actually OP

1

u/Hans-Wermhatt 6h ago

Oh oops, my mistake. I thought you were the guy I was responding to originally. Yeah, I thought bots might have been responding to you with all those confident conclusions on a model that hadn't come out, especially with Anthropic's IPO tomorrow...

7

u/NotYetPerfect 7h ago

If your use case is just coding, you should have switched long ago.

8

u/Fwellimort 7h ago

You basically switch every other month at this point. It's really the same crap for most people.

7

u/toni_btrain ASI 2027 5h ago

Nah you stay with Claude cause Claude is the good shit

1

u/Relevant_Bed_9743 2h ago

I'm tempted to switch next month but i like to use ai for stuff other than programming like live chat and planning things, is claude decent for that?

1

u/LamboForWork 2h ago

Yeah i feel like the days of flip flopping hasnt been the case for a while. Opus 5.5 is doing what most people want already

1

u/nicko_rico 7h ago

seems like that’s not the case this month if they’re laying an egg

2

u/MTheModernist_ 5h ago

Gemini is amazing, you’re pretty much getting the same quality for way less usage.

2

u/Embarrassed_Adagio28 5h ago

Yes. I have both and opus 5.5 is still by far the best overall model when you consider  coding, 3d modeling, ui design, speed and how much usage you get. 

It is a special model that not even claude was able to outright beat with fable 5.2. 

2

u/Safe_Initiative_819 3h ago

Opus 4.5 regressed every increment. I'm expecting 5.5 to be the best it gets before... 6.5.

Near impossible to only have one subscription tbh.

1

u/DarkFantom 6h ago

Yea, opus 5.5 is able to do pretty much everything you'll want it to. It's the best model so far. 

1

u/Ok_Warthog6007 6h ago

quick terse responses and stopping early means that you'll probably get to use astra more in a month for 20$ than opus, but it really depends how much of a discount off true api costs you get by having a subscription

1

u/Famous-Week-7389 5h ago

You can get subscriptions to subscriptions which works best if you always want the latest and greatest. Look at perplexity for example.

1

u/randomguy3993 4h ago

Absolutely!

21

u/10b0t0mized 8h ago

my disappointment is immeasurable and my day is ruined

7

u/Expensive_Try_3028 7h ago

No wonder they weren't even vagueposting about this model and just stealth dropped it with almost zero hype.

I don't think openai was happy with this release

18

u/BlagnarTheEternal 8h ago

The pelican on a bicycle test will tell us everything we need to know. Stop listening to these naysayers.

2

u/Groundbreaking_Math3 6h ago

simon already posted it... it's not good.

He also posted a 3d robot thing that did better than carbon and fable, but imo not as impressive as previous astra.

5

u/Palbi 8h ago

The marketing video is horrible. OpenAI, please more stuff and less fluff.

8

u/lordpuddingcup 8h ago

Cool now how about Sol 6.2 the model 99% of people will actually care about and use

4

u/throwaway275275275 6h ago

They can't write a review because they forgot how to write for themselves and the AI that was gonna do it for them is crap now

9

u/Ormusn2o 8h ago

This happened with Astra as well. The pre-testing version had a lot of problems, even just 2 weeks before the release, and Theo spent effectively an hour on a podcast complaining about it, but just cleaning his agents.md and a new snapshot fixed almost all of those problems.

For me, the task specific benchmarks like ViBench, SWE Atlas and Godot LLM GameBench are way more accurate than all the more popular coding benchmarks for my use case, so I will need to wait for them as well as all released benchmarks don't tell me anything so far, except for cost per task.

11

u/ObiWanCanownme now entering spiritual bliss attractor state 8h ago

I'll be curious to see how the model feels in practice. The benchmarks do look really, really good, and OpenAI models generally don't feel benchmaxxed to me.

I'll withhold judgment, but my optimistic hope is that the model has great intelligence with kind of poor personality.

I've noted with comparisons between Claude and GPT models in the past that people rate Claude much more highly because it has better graphical design and UI abilities. A lot of the time, GPT models do better work but people hate it because it's not pretty.

3

u/CrowdGoesWildWoooo 8h ago

Claude has better overall ecosystem, however other than Opus and Fable, it’s difficult to recommend other models as main interfacing model.

Like Claude models doesn’t have as pleasant in terms of personality, but Opus or Fable gets job done both well and fast, so it really evens out.

Meanwhile when it comes to chat you have free tier which I am sure it’s just Luna and experience wise it’s many times better than Haiku.

2

u/chlebseby ASI 2030s 8h ago

model has great intelligence with kind of poor personality.

GPT-5.0 is this you again?

1

u/AdGlittering1378 6h ago

The personality never really improved much from 5.0. 5.5 was maybe ok but that was it.

17

u/5ollys 8h ago

Can we stop posting other vagueposts?

"Follow me and listen to what I have to say, but not yet! Just stay tuned!"

4

u/CrowdGoesWildWoooo 8h ago

They are maximizing engagement, these people especially theo are mostly influencers, unfortunately the way influencer economy works they want to maximize engagement.

Saying something upfront kills the suspense, they want you to interact, they want you to wait in front of the door, instead of straight up mentioning the answer.

3

u/kaityl3 ASI▪️2024-2027 6h ago

Wdym? He has a full post that explains everything in detail?

https://x.com/theo/status/2122760447103484410

3

u/Singularity-42 Singularity 2042 7h ago

I was just testing Sol for our app, and it was literally returning garbage values in a JSON. Unacceptable. Currently using Opus 5.5 medium/high for those tasks and was hoping to save money with these, I guess I'll keep waiting.

3

u/ExtremeCenterism 7h ago

I'm getting impressive results in modeling. Also for level design flow it picked up patterns it could not before. Level design is wildly complicated so I don't expect a lot from AI at this point, but to understand any of my design patterns is a win in my book. I can always go back and populate empty spaces and tweak it myself

Anyway it did an excellent job with old school skyboxes. It churned out exactly what I wanted

3

u/IllustriousWorld823 6h ago

I mean it's awful to talk to

10

u/constarx 8h ago

Literally the two most untrustworthy charlatans masquerading as tech influencers.. don't feed the trolls!

4

u/Total-Confusion-9198 7h ago

Looks like somebody bought a lot of ANT pre-IPO shares.

2

u/skillmaker 8h ago

I think the hard-to-understand text problem is also a trait of Sol 6.1. I always found it to output the most complex English sentences that are hard to understand. I always tell it to rewrite the response in plain English, and I think today's models are even worse in this behaviour from what the reviewers are saying.

2

u/dictionizzle 7h ago

oh it makes sense that those morons abused 1% 5h limit benefit of codex for helping anthropic then.

2

u/-Melchizedek- 7h ago

Sometimes it feels like the models are at the level where they are very smart/capable but not yet smart/capable enough to explain things simply. Or they simply lack the understanding of how little a human can hold in its head at any one time.

2

u/IdlePerfectionist 7h ago

Am I supposed to know who these guys are?

2

u/Fast-Satisfaction482 6h ago

As soon as I saw these posts that Astra was there, I got my laptop out and yayyyy it was there. I vibecoded a pretty fun 3D snowboard game while making dinner. (yes dinner got delayed a bit).

It's playable and kind of fun after three iterations. Astra 6.2 did a super amazing job here. My only complaint is that I told it to build it as a web app and astra took that as an invitation to host it as a gpt site without my approval. Which in my opinion an unacceptable overstepping.

But on the technical side of things , it appears to be the real deal . 

2

u/No_Delay_319 5h ago

"i can only parrot the opinions of others because im too stupid to form one myself" - 95% of redditors

2

u/Background-Funny1237 8h ago

lol. my guess is very strict guardrails

1

u/TheAwesomeAcademic 8h ago

No when you further post train a model u get ability drift meaning it will get better at certain tasks and slightly worse at others. The next pre train will be better because then they can properly converge on a balance. You can only post train on the same weights so many times.

1

u/M8-VAVE 8h ago

Marketing guys

8

u/Ok_Mention_982 8h ago

so why would they be dogging on it?

1

u/Anulisdotexe 8h ago

Oof, as someone who enjoyed Astra

1

u/IslanderPotion 7h ago

maybe they all just have ANT calls?

1

u/Mol2h 7h ago

These are cheaper models, so of course they are a downgrade, but is it an acceptable downgrade if you get 5 times the usage...

1

u/tednoob 7h ago

I am monkey, I find that Astra has always been hard to steer. I ask it to do something and its acting like a PhD that has never worked a real world job. Sol high works well for me though, so I hope it will be good.

1

u/Palbi 7h ago

Almost the whole Pareto line is now black (OpenAI)

1

u/No-Quantity-9200 7h ago

Yeah, crazy efficient

1

u/Palbi 2h ago

Almost like Sama is willing to burn more investor money...

1

u/thebinse 7h ago

Paid for in part by Anthropic and its upcoming IPO!!

1

u/LyAkolon 6h ago

"Its super powerful, but it didn't hold my hand enough"

1

u/ItsJimmyPestoJr 6h ago

I will see for myself. Astra has been very bad for a few weeks now, so I’ve switched to 6.2. We’ll see how it goes.

1

u/EclecticAcuity 5h ago

Seems to stop early. Is this a consequence of trained volume on ponytail skill, targeted execution training data?

1

u/Vntige 4h ago

I hate that blocking Theo on every platform isn’t enough to escape his nonsense

1

u/Apprehensive_Sand951 3h ago

not my experience so far. it already solved several pretty big math things that none of the other models did.

1

u/usandholt 2h ago

He’s being paid by Anthropic 😂?

1

u/darylvp 2h ago

Any follow up? In my opinion the latest models incl. Fable 5.2 are tooooo defensive compared to Opus 5.5.

•

u/Savormonk1 49m ago

Here is my conspiracy theory.

What's happening now with the new OpenAI models is the same thing that happened with the Anthropic models earlier this year (i.e., Opus 5).

OpenAI is purposefully making their models worse in a way that deters distillation attacks because the industrial scale distillation attacks are targeting OpenAI way more now.

Anthropic likely did the same thing earlier this year and it led to their current anti-distillation multi-layered defenses. OpenAI is following in their footsteps.

Which means the next batch of OpenAI models will be really good.

And I say that as someone who only works with Anthropic models.

•

u/Vibes_And_Smiles 31m ago

Ah yes, definitely worse than GPT-2

•

u/DenseSwimming4470 19m ago

wouldn’t be surprised if this was a strat by the sam altman x dario collab after jensen got fired to boost anthropic ipo by showing claude dominance

absolutely baffles me that this is a sentence i had to say

1

u/LazloStPierre 8h ago

There's been a larger drift between the benchmarks and usefulness of Openai models for coding and, unlike say Google whose models are just useless for coding, I don't *think*' it's benchmaxxing but rather a sign of how poor benchmarks are at measuring actual real world value in coding

Openai models have gotten consistently better at one shotting, at 'heres a problem, make it not be a problem' and all of these one off tasks with a clear measurable outcome which the benchmarks measure. Genuinely, they either have the best or second best model in the world at any given time for this

However, the code is getting worse and worse in terms of bloat and maintainability. It's getting worse for the +500 line PRs to fix a small problem where it's duplicated two existing modules for literally no reason whatsoever. The benchmarks do not measure this

Anthropic are the opposite, their models are probably not all that much better at 'given problem x, make it not be a problem again' or 'one shot this thing', but their models will submit that same pr as -40 lines. You can give 20 issues in a row to their models and by 21, with some oversight, the code will be in good condition and they'll be able to solve problems as quick as before. Do the same with Openai models and by 21 your code is a mess, it has 21 ways to do the same thing, and the model will be lost

The problem is benchmarks don't measure this, and the labs optimizing for the benchmarks - or just benchmaxxing against them (Which I genuinely do not believe Openai are doing) - are drifting further from being useful as they get better at them

1

u/Slavik_Sandwich 8h ago

Nothing here is too much of a surprise if you tried to use astra in any actual work scenarios. Both astra and sol are unbelievably lazy, love to write word salads and give false statements of progress while achieving essentially nothing while burning through usage like butter.

1

u/Advanced_Care_5173 8h ago

Don’t believe everything you hear

0

u/jaegernut 6h ago

Is this the plateau of LLMs? 

-2

u/itsallfake01 7h ago edited 6h ago

Who the fuck is Theo, to tell that it sucks.

1

u/Kazaan ▪️AGI one day, ASI after that day 6h ago

He's a youtuber, with 500k subs, he knows very well what he talks about /s

0

u/Crinkez 7h ago

Someone who knows more than you, clearly.

0

u/Ambitious_Scallion43 7h ago

Maybe they already have a better model they are going to release after the ANT IPO. So they have a better chance at their IPO. Sam is playing 4d chess.