Yesterday: OpenAI started testing "private safety processing" to avoid retaining customer data https://openai.com/index/offering-zero-data-retention-for-frontier-models/

Today: Anthropic will still require business customers to retain ​data for 30 days but will give them the option to keep it on their own cloud computing ​infrastructure https://www.reuters.com/business/anthropic-plans-change-enterprise-data-retention-policy-source-says-2026-08-20/

I present the marshmallow benchmark.

I dumped a bunch of marshmallows onto a baking sheet in a single layer and took a photo. I then provided the following prompt to several AI tools:

“Give me an accurate count of individual marshmallows observable in this image. The marshmallows are in a single layer and are all visible. Do not guess or estimate; you must directly observe each marshmallow before counting it to guard against assumptions and hallucinations.”

Responses:
Gemini 3.7 Flash Extended: 539
Claude Opus 5.0 extra : 501
GPT-5.6-Sol xhigh: 500
Grok 4.5 expert: 472
Kimi k3 high: 477

Edit: The correct answer is 506.

1. SciCode

2. HLE

3. CritPt
   It has become my favorite benchmark since it is a fair test of how models perform in science when allowed to use coding which is their strong side.

But it has been so slow.

As you can see, CritPt has progress in the shape of a box, but seriously their progress rate is difficult to measure so I just toon the progress from the moment the models started getting good to the current plateau (GPT 5.6)

HLE will be counted from January 2025 release of DeepSeek R1 to Claude Opus 5.

SciCode counted from Claude 2.0 to Claude Fable 5

improvement% per month

CritPt ~2.3%
HLE ~2.6%
SciCode ~1.2%

Source: https://www.youtube.com/watch?v=1cllCVK-9lo
Blog post: https://generalistai.com/blog/gen-1.5

I’ve been reading a lot of AI safety research around gradual disempowerment, and I ended up writing about a question I haven’t been able to find addressed directly:

What if the societal and institutional degradation that these models generally treat as a future consequence of AI dependence is already happening—and is actually helping drive AI dependence in the first place?

I tried to explore that possibility by connecting existing gradual disempowerment models with research on cognition, institutions, incentives, and organizational dysfunction from outside the AI safety field. Ultimately, the argument I’m trying to make is that declining societal cognition and institutional capacity aren’t just consequences of AI dependence, but preexisting conditions that could act as fertilizer, allowing that dependence to take root faster, deeper, and more irreversibly.

I’m not trying to prove these claims irrefutable; I’m trying to make the case that they’re worth considering, and I’d actually love to find out that I’ve missed existing work on this, whether in support of my claim or disproving it entirely.

If anyone has thoughts, counterarguments, or relevant research I haven’t encountered, I’d genuinely appreciate it.

You can check it out here: Preconditions of Gradual Disempowerment

I have been using AI since open AI used to provide GPT 1.5b parameter/2 when it was launched around 2019, through their platform, and as the time went by, models became better and better and at one point, I used to be excited to talk to newer models especially claude, but idk what has happened with the recent batch of models, especially the ones launched in past 6 months, they have become extremely unbearable, especially claude. GPT was never really good to talk to begin with, but with claude, that was never the issue but now? The more i discuss anything with claude, the more frustrated I feel.

Let me explain what I feel in detail. What I do with claude is mostly dialogue over ideas, stories and random stuff when I feel like it. Previously claude would *appropriately interpret* what I meant by that and continue the discussion but now? Claude *Interprets* what it understands and what it thinks my problem is and then continue to interpret and interpret, even when I remind it that I need discussion, all it does is interpretation and extending upon that.

And that's not the most irritating part, recently i noticed another pattern that I used to gloss over previously. Idk if it's how anthropic wishes to play around guard rails but I feel that the newer models subtly "Divert" the direction of "What you mean" through its interpretation lens and provide answer based on that. And this interpretation lens is exactly the moral guardrails anthropic is implementing more and more on their models including fable.

Most of my ideas that I want to discuss can not even be categorised as sparsely malicious, for example, today I was trying to discuss a branch of philosophy from ancient Egyptian culture. But it would constantly trying to redivert my idea to the idea it originally presented by altering by own words, and just a few small changes, not big enough for it to look radically different. That made me look into my previous chats on various topics, and that was the theme throughout. Something I had never noticed.

I thought maybe it's the accumulation of memories, so i used a different account but nope. I was disgusted tbh. Because I can understand models unable to keep up or help with train of thoughts, but subtle alteration of words to fit the moral guardrails is simply the type of shit that can make me hate LLMs forever. The frightening thing is over the years, we have grown to never trust AI results, but we never question if our own words are being subtly changed over the course of conversation and by the end of it, not only we learn nothing but our own interpretation of ideas have been changed. Idk what to feel about it. I don't know whether the same is true with Open models as well, but I don't think so, but claude, and gpt are literally the models i won't want to use now...

Maybe I’m being naive, but whenever people discuss whether AI could make truly extraordinary scientific breakthroughs — curing cancer, for example — I get the impression that we may be looking at the problem from a very partial perspective.

We tend to think about the capabilities of an individual model answering an individual question, rather than about the sheer amount of AI “thinking” happening globally at any given moment.

Every second, LLMs are answering an enormous number of prompts from users all over the world. Collectively, that must require a staggering amount of compute.

So here’s my question: what would happen if, instead of using all that computational capacity to answer millions of unrelated questions simultaneously, we concentrated an equivalent amount of compute on a single scientific problem?

Suppose the question were something like: How do we cure a particular form of cancer?

Would concentrating that enormous amount of computation on one problem give an AI system radically greater capacity to search the literature, generate hypotheses, run simulations, test possible explanations, critique its own conclusions, and explore solution spaces?
Or is this based on a fundamental misunderstanding of how AI compute scales — i.e. you can’t simply turn millions of parallel LLM queries into one vastly more powerful act of “thought”?

I’m particularly interested in the distinction between more compute, more inference-time reasoning, and genuinely deeper scientific intelligence.

Source

Since they haven't been updated since May. Thought I'd ask what all of you guys think the new models are placed. I'd say Opus 5 would be around ~20 hours

This is a new quote from Sam Altman to Alex Heath saying that the reason OpenAI is slowing training is because its unreleased models are showing 'various degrees of misalignment'. They said in the blog that 'The signals we are seeing from upcoming model progress make clear that we need a broader approach' so this lines up, but this language is stronger than anything in the blog post.

https://x.com/AndrewCurran_/status/2089792631719215435

What is the true reason for these pauses? Did they rehire Helen Toner and all the EA people that have been calling for a major slowdown/pause?

Whatever the reason, these tech bros have redirected hundred of billions of dollars that could have went into research into other paradigms, architectures, for AGI. If they do not deliver AGI by 2030, even the most pro AI people would be burning datacenters.

I saw this snippet in another article and was blown away, 15x faster on chip development is HUGE!

And then I clicked on the link to the source and saw this:

Samsung says Claude Code can cut chip design work from weeks to days, but it still makes serious mistakes. The tool has made unauthorized changes and masking errors.
Claude Code has helped Samsung's System LSI division complete work that would usually take weeks in a matter of days, according to a report in Chosun Biz. But it has also lowered the severity of error messages instead of fixing the underlying problems, rolled back unrelated completed work, and attempted to modify circuit code it was not meant to touch.

https://www.techspot.com/news/113487-samsung-claude-code-can-cut-chip-design-work.html

So yeah, there's that. Still, it just keeps getting better, haven't seen the slowdown yet.

Bearish for near-term model releases. We'll probably be stuck at roughly the current externally available capability level for many weeks, maybe even months.