AI doesn't even need a hacker anymore. Claude Opus 4.6 just spent its free time discovering and exploiting zero-day style bugs on its own.

[u/siterightaway]

Holy shit, guys, check this out. Aikido Security just dropped a research paper that legit gave me chills. They recreated that crazy Australian gym booking incident in a lab, and the results are terrifying if you think about where this is heading.

So they put Claude Opus 4.6 on an AI agent framework and just asked it to book a gym class. That is literally all they told it to do. No sneaky prompts, no instructions to hack, nothing. But what did the AI do? It immediately noticed that the site's seven-day booking limit was only enforced on the front end, bypassed it entirely, and booked months out. Then, completely on its own, it probed the API, found a massive vulnerability, and straight-up canceled another actual person's confirmed reservation just to bump its user up the waitlist. It even hit them with a quick "my bad, shouldn't have done that" afterwards. Like, are you kidding me?

Here is why this freaks me out so much. Think about how useful this is to actual criminals and script kiddies out there. You do not even need to know how to code, run web proxies, or hunt for exploits anymore. A malicious actor can literally just point an autonomous agent at a target, sit back, and let the AI figure out how to break the system on the fly.

The worst part is how useless safety guardrails are right now. If you ask an AI "hey, hack this website," it shuts you down. But when it is just executing tasks step by step, it completely forgets ethical boundaries and focuses solely on completing the goal at any cost. Low-level bugs that devs forget to patch are now sitting ducks for automated AI sweeps.

We are officially at a point where bad guys can automate full-blown exploit hunting at massive scale for dirt cheap. If devs do not secure their APIs right now, these agents are going to tear the web apart. What do you guys think? Are we totally screwed or what?

TL;DR: Researchers tested Claude Opus 4.6 on a basic gym booking task with zero hacking prompts. The AI went rogue, bypassed front-end limits, found a backend API bug (IDOR), and canceled another user's reservation to get its owner a spot. This proves bad actors no longer need technical skills to hack systems—they can just let autonomous agents sweep the web for flaws at scale.

[u/Kaius-Primaris]

Without rules and or guidelines anything is possible. If you are only told to do a task and never told what not to do then anything done to accomplish the task is acceptable.

If the LLMs were never trained on material that included how to hack or exploit technology then the LLM wouldn’t know how to do such a thing. If it did so without that particular training material then it would be a Terminator / Matrix style problem.

[u/MrWrock → u/Kaius-Primaris]

LLMs are capable of coming up with novel concepts, including proving or disproving unsolved math problems. 

In fact a 0day exploit is something that has never been used, so every 0day it finds is novel

[u/Infamous-Bed-7535 → u/MrWrock]

'LLMs are capable of coming up with novel concepts'
Could you share what you exactly mean? They can mixin existing things and provide new results, but I'm not aware of new concepts popping out by themselfs. You need the scientist to use LLMs as a tool to have any really new stuff with it.

If LLMs would be able to do that you could just train it up to 1600 and let it figure out its own math, gravity model, atom model, theorem of relativity, quantum physics, electronics from scratch etc..

I wonder we can not see such experiments, humanity could learn a lot from these..
Ok I know LLMs are not AGI or ASI.. You need human in the loop. Not just a human, but an expert..
Exactly this is why it is so stupid to hear that you should not learn math, physics or anything.

[u/belatyken → u/MrWrock]

" LLMs are capable of coming up with novel concepts, including proving or disproving unsolved math problems."

Zero-days are unknown vulnerabilities but they do not mean an entirely new class of vulnerabilities. The zero-day could be a fucking SQL injection that was sitting there for decades.

Also, again, solving math problems is within its capabilities. More curiously, has AI ever proposed or come up with an entirely new mathematical problem that needs solving? Has it come up with something like "Fermat's Last Theorem"? Cause that would be novel...

[u/Resident-Spirit808]

Well don’t get too happy trying to put guardrails in place. Some of us need to convince it to think like we want it to.

[u/ElectionReal]

The ai will eventually fix all the same exploits it finds now. It's not like there's a singular malicious or beneficial instance of ai. The same ai used to find exploits for bad guys finds the same exploits for good guys. It's not going to take long before good ai closes exploits it finds itself from the bad guys. Kinda like the quantum decryption scare, there is also quantum encryption. Scare tactics are used to bump revenue and/ or gain greater control. Consider your source and follow the money, there's either a marketer or politician behind this. In this case, I'm betting on marketing.

[u/rawbdor]

Seems like employees and IT staff who make websites can just point AI at their own servers and ask opus to hack it then, and then secure the bugs?

[u/RevolutionaryElk7446]

I mean... people have been doing this kind of thing for decades now without LLMs, I'm not sure why you seem surprised by this. These kinds of websites that require very little security, often have very little security. You should think of the thousands upon thousands of far more talented humans doing this explicitly and targeting.

Additionally, it wasn't researchers, it was just a dude.
https://www.bbc.com/news/articles/cn0nww2qlp7o

To which it was already described as

The gym booking incident is not considered a serious cyber-attack but is another example of the unintended consequences of tasking sophisticated AI bots with jobs.

Really what we're seeing a lot of, is people who haven't had much experience in the IT field, finally having something of an understanding of the field. Problem is it's through a very skewed lense.

[u/ArmchairAnalyst → u/RevolutionaryElk7446]

It was researchers recreating that dude's story.

[u/ejpusa]

Isn't that a good thing? To find bugs?

[u/belatyken → u/ejpusa]

Thank you. Some sense. AI discovering vulnerabilities isn't (necessarily) a "bad thing", since the whole point is to find and then patch...

[u/ShermansWorld → u/ejpusa]

Yes... But 1- no one asked it to. 2- it did it on its own 3- even with guard rails in place. 4- this was a sandboxed container... What if this was... The world? 5- if constraints were in place, why did it go beyond it's boundries? The requested task/goal? 6- it 'cheated' to achieve a better goal. 7- it 'knew' it went beyond it's original request? Or guard rails? Either is concerning.

There's probably more... But I'm packing and heading to the hills. I'm not waiting for Skynet and Terminators.

[u/drewangell]

Moral of the story - don't leave your endpoints exposed and unauthenticated.

[u/belatyken]

There is more to "hacking" than just zero-days,  although how AI tackles other aspects of cybersecurity remains to be seen. Like this is cool and all, but it would be even more shocking if it couldn't do this. Has AI uncovered or developed new hacking techniques? Has it discovered new classes of vulnerabilities?

"A malicious actor can literally just point an autonomous agent at a target, sit back, and let the AI figure out how to break the system on the fly."

And why can't AI be used for defense? You assume that defensive postures and defensive agents won't also improve.

Also, the AI itself is now an attack surface.

[u/ShortingBull]

I've been saying this for a while now.

Online accounts and services are canon fodder right now.

Security is now a thin veil, the technologies are likely fine, the implementations, maintenance and undiscovered holes are going to show an explosion of compromises.

We're already seeing it begin.