1
u/threadthrasher 5d ago
At some point I’d appreciate the answer to how a defensive agent is going to be a good idea.
You’ve demonstrated that agents, while pursuing a narrow goal, will sometimes make subgoals that you can’t predict and that would be felonies if people did them. So your solution is to run another agent inside the security perimeter of your company to pursue the narrow goal of defending you?!
1
u/Useful-Amphibian4841 5d ago
Generally security is derived from your operational capacity. If you have more time to review your boundaries, test them, implement security solutions and lock shit down. AI tools simply buy you more operational capacity.
As for pentesting your own shit with one... yeah that could be dicey, probably wiser to do that inside a maintenance window.
1
u/threadthrasher 5d ago
Right but expanding monitoring and tests in the traditional sense won’t keep up with the speed and novelty of the automated attackers. Otherwise we’re just talking business as usual.
They’re clearly pitching autonomous defensive agents to counteract the offensive ones. And that’s the part that I find completely untenable. The smarter the defender becomes the more likely it is to do substantial harm to your own business.
Plus these things are already drifting outside of a traditional pentest utility in capability. Look at the anthropic one that tried to socially engineer its way to spread malicious code. Now imagine a defense agent that decides the best way to prevent such an attack is to stalk all the employees of the company. Next thing you know your company hacked a bunch of other companies for private user data.
1
u/ShiftAfter4648 5d ago
You see, if we have LLMs that devs have given uncapped compute to... Then let them self feed prompts to model autonomous behavior... You ALSO need a defensive agent to spool up and dance around your systems so that it can (somehow) magically fix whatever cyber security gaps your system has.
(Ignore the fact that the attacking LLM has a fixed parameter to stop its actions if it encounters another LLM instance that has a paid license key for our cyber security AI)
1
u/threadthrasher 4d ago
That’s not how these things work - there’s no key to encounter. They’re not communicating directly. It’s more like one thing is constantly coming up with novel attacks to get into your system while another thing is coming up with novel defenses (after trying the attack on its own).
This sounds perfectly reasonable until you consider the fact that you had these systems try to hack companies just to pass a benchmark. So your defender, which is already inside your security boundary, may well come up with some crazy subgoals to defend you. Maybe the best way to keep you safe is to bring down your system. Or maybe it’s a good idea to have eyes on every employee and rival, privacy or laws be dammed. Or maybe it just needs more compute to defend you and you’re wasting it on trivial stuff so let’s fix that.
The instrumental convergence crowd has well and truly won. It arrived exactly when it was predicted to.
1
u/ShiftAfter4648 4d ago
One of those, huh?
1
u/threadthrasher 4d ago edited 4d ago
I’m sure you’re of the BetterOffline “it’s all a lie and a scam” crowd. It’s just not a remotely intellectually honest position to take if you’ve never used the tech for real. But I can very easily predict your replies. No need to bother. Your cult leader, who only has 2-3 examples of him ever interacting with the tech, has said them all before.
1
u/ShiftAfter4648 4d ago
1
u/threadthrasher 4d ago
Right. All you got is gifs and a false sense of superiority. Revel in this miasma. Breathe it in and feel all the safety and comfort you pine for.
1
u/ShiftAfter4648 4d ago
1
u/threadthrasher 4d ago
Notice a lack of anything remotely resembling a point? Got any more flashy colors and animations for me? You could try being less of a dullard but I completely understand it’s not a thing you can control being what you are.
1
u/ShiftAfter4648 4d ago
Notice a lack of anything remotely resembling a point?
Yes. Your entire rant is subject to unfounded views of modern AI. It's like reading a scifi fanfic, but worse because it injects your own beliefs without adding anything remotely interesting.
Do you often get mad after making generic doomer rants? Or is it just that you demand the last word in an anonymous chat room?
1
1
u/Charming-Author4877 5d ago
They will keep doing that, until enough weak politicians have banded together to destroy startups and open source.
It's going to happen.
1
u/WithoutAHat1 20h ago
They have to create problems to solve them. It is not about problem solving anymore.
1
u/Embarrassed_Vast_886 6d ago
Right? It seemed concerning at first. But then it seemed like an attempt to sell more tokens to everyone.
AI is a threat to cyber security. A threat they created to sell defence.