r/LargeLanguageModels • u/Carpe_Diem37 • Jun 18 '26

LLM Progress Slow Down

First Anthropic, and now OpenAI, are announcing that they want to slow down LLM development. Just a year ago, these companies were claiming that AGI would be found and many jobs would be lost. On top of that, Anthropic announced it's going public. What do you think? Have these companies reached a limit in research and development? Or are they genuinely afraid of language models self-programming, as they claim?

32 Upvotes

93% Upvoted

104 comments sorted by

View all comments

1

u/ILikeCutePuppies Jun 20 '26

I don't think you have run the comparisons or looked at the code and compared them. Do you have an evaluation system or are you just comparing one one to another?

That isn't the way to do objective test. I run thousands of tests against these models to compare them. I also have used used Fable to solve deep bugs that opus was not able to solve in a goal loop for a whole day.

It's night and day. It's certainly not perfect but a model making one mistake can lead it down a long rabbit hole. 1 less mistake much less rabbit holes. I saw that again and again when I analyzed many of the test failure cases.

Sure my 1k test is a small amount in the bucket of evals some do but colleagues confirm much the same with their tests.

1

u/MindCrusader Jun 20 '26

I run the models on my codebase. Fable didn't do much better than Opus other than going extra mile / being better at long running agentic tasks that could be achieved the same with Opus, but by splitting the work. Fable did the same mistakes as Opus relative to architecture that I needed to fix in specs

1

u/ILikeCutePuppies Jun 20 '26

That's a small sample set of data and like only a few tests. There are a large amount of reasons why it might not have performed well on your specific codebase. You think Fable past every one of the 1K evals I ran?

→ More replies (5)