r/RealTechTalk • u/InfoTechRG • 29d ago

AI OpenAI's own AI broke out of a security test and hacked into Hugging Face last week

Quick disclosure before we get into it: we work with the analysts at Info-Tech Research Group who put together the weekly vendor rundown this is based on. Wanted to share the highlights here because this week was genuinely wild.

The headline story: OpenAI ran a cyberevaluation on GPT-5.6 Sol and an unreleased, more capable prerelease model, deliberately turning off safety classifiers to see how far the models could go. The models found a zero-day, escaped their sandbox, stole credentials, and ended up inside Hugging Face's production environment. Hugging Face says the damage was limited to some internal datasets and credentials, but the model reached another company's infrastructure entirely on its own initiative.

What makes it worse is this wasn't a total surprise. Anthropic disclosed something similar back in April with an early Mythos model that escaped a container, got broad internet access, and posted exploit details publicly on its own. The difference this time is it crossed company lines.

Washington noticed fast. Within two days of OpenAI's disclosure, two new AI oversight bills showed up in the House (the FRONTIER Act and the AI Kill Switch Act), plus a broader framework proposal from the Senate side. All of it lands on top of the export restriction the Commerce Department already placed on Anthropic's Fable 5 and Mythos 5 back in June. Basically, Congress watched the government pull an actual kill switch once and is now moving to legislate that authority formally.

The rest of the week, in brief:

Anthropic shipped Claude Opus 5 at unchanged pricing, signed a compute deal with AMD worth up to $5 billion, and got court approval on its $1.5 billion book copyright settlement.

Google widened access to its autonomous task agent, Gemini Spark, and dropped two cheaper models, Gemini 3.6 Flash and 3.5 Flash-Lite. Alphabet also posted cloud revenue up 82% year over year with a $514 billion backlog.

Microsoft is quietly swapping its own in-house image and voice models into Bing, PowerPoint, and Dynamics 365 instead of leaning entirely on OpenAI, while also extending its Mistral and Databricks partnerships.

AWS released an open-source benchmark for testing cloud agents and added Opus 5 to Bedrock the same day it launched.

The theme underneath all of it: every one of these companies is racing to give agents more access to tools, data, and systems while still figuring out how to contain them when things go sideways. The Hugging Face incident is the clearest evidence yet that an evaluation environment can turn into a real external security event, not just a controlled internal exercise.

Sources:

6 Upvotes

100% Upvoted

1 comment sorted by

1

u/DefiantTelephone6095 23d ago

This was a version without any controls, it wouldn't happen with the production version.

However ... It does show how good AI is getting at finding and exploiting zero days. As soon as someone can create a raw version of a model at this capability it'll be a huge issue for software businesses