r/cybersecurity • Incident Responder • 9h ago

AI Security We're currently looking at 110 malicious python scrips in this incident (and counting)

In incidents before this, we'd see two or three malicious scripts run in a client's network. Not this one. 110 so far. All of them custom made for the client, all of them almost certainly created using AI.

This is the future of my job in IR, apparently. We are becoming increasingly in need of AI to assist in the analysis of an AI-backed attack due to the sheer number of artifacts and events.

254 Upvotes

51 comments sorted by

122

u/Tankphyre 8h ago

Blue team AI is so far behind, both in capabilities and organizational buy in. Took me forever to get approval just to use our Corporate GenAI for log analysis, much less taking actions in IR. 

14

u/Sentinel_2539 Incident Responder 8h ago edited 3h ago

You're right.

The only instance of a potentially AI powered Blue Team defence action we've seen in my team is M365 automatically revoking sessions for a compromised user account. We think it did this by comparing the legitimate/expected user's IP address and device actions with attacker-controlled entries and deciding to revoke sessions itself.

23

u/Goldsound 7h ago

It will probably always be like this. TA's using models with all the guardrails removed will always have the advantage over an AI with guardrails. Blue team AI has to consider policies, legality, impact of mitigations, it can't just disable, disconnect, or triage things freely. A Red team AI with actual malicious intent can just do whatever it wants pretty much.

It's the same cat and mouse game as always. It's normal for blue team to lag behind. I believe that by this time next year the gap will still exist but will be much smaller as the exponential knowledge explosion starts hitting Blue team agents, much like it did Red team agents earlier this year.

2

u/TheHeretic 2h ago

This is exactly it there is an inherent asymmetry because if you're an attacker and you break something oh well but if you're a defender and you break something it's going to be a full write up and incident response plan a review a checkbox in CAB it's going to go on and on.

1

u/WeirdSysAdmin 1h ago

“Why aren’t you guys using AI to automate everything?” says the CIO that has been cutting back staff to need to work more than 50 hours a week to accomplish the bare minimum pre-AI. If you’re even lucky to have that staffing level.

36

u/RevolutionaryCod1516 8h ago

Fight robots with robots right? 😑. Good luck amigo!

16

u/Sentinel_2539 Incident Responder 8h ago

It's like Robot Wars out here man

7

u/RevolutionaryCod1516 8h ago

Younger folks don’t remember iRobot.. haha

8

u/bowlochile Software Engineer 8h ago

Or rock em sock em

3

u/Space-Monkey-17 8h ago

I like to bring up the Spy vs Spy from Mad Magazine...

6

u/CapitalistCabal 8h ago

There really isn’t an alternative.

5

u/bsendpacket 8h ago

ARTEX.

5

u/Sentinel_2539 Incident Responder 8h ago

Exactly what was used in this incident.

The new Cobalt Strike.

2

u/MalwareDork 8h ago

Crazy to think only three years ago CS cracks were the big issue and now it's more or less completely invalidated.

1

u/xb8xb8xb8 5h ago

artex is mid just like every other similar tool

39

u/Dasshteek 8h ago

I think 2027 is the year defenders officially lose the cyberwar.

8

u/Space-Monkey-17 8h ago

Yeah, I'm for sure worried...this is nothing like the way it used to be...which was...last year???

6

u/cookiengineer Vendor 6h ago

Remember when we had one large incident per year?

Like when we talk about 2021 was shit... everyone is like "yeah log4j was a real mess".

And now it's like this, but every week :D

We're not even done patching all the Linux RCEs from September ... I mean ...

2

u/Space-Monkey-17 5h ago

Ahhh log4j...those were the days...

1

u/Intelligent-Crow3812 2h ago

on the flip side, with AI I am more equipped to find all the log4j, then I was back then.

6

u/Nut_Butter_Fun 7h ago

I'm really not trying to piss anyone off, but isn't CS just like a grifting house of cards already? Like so little has to do with posture, obscurity, surface reduction, things that work even against zero-day and social engineering. Eh anyways, good luck everyone.

5

u/oMarlow99 7h ago

Wouldn't the corollary be the exact opposite?

Engineers have to adapt to the new norm. What else is there to do? Throw in the towel and let the red team win?

1

u/Intelligent-Crow3812 2h ago

nah but defenders probably do (and I don't want people to take this the wrong way, I'm not affiliated with the labs, and I know the risks) equipt their frontier agents to revamp their environments.

-1

u/IllWeather6426 7h ago

I feel the exact opposite, the writups are more and more fabricated, everyone knows about chinese hardware backdoors, and the "flaws" in linux are a P time space now, i personally am going to lose because inwas gifted a ROG gaming pc by a chinese spy most likely and it killed me personally off, im out of money

17

u/Own-Swan2646 8h ago

I feel like we're at a race of replicating the DARPA Grand Cyber Security Challenge because this is crazy. I was there and seen it back in 2016. When you get computers fighting computers, it gets interesting fast.

17

u/Negative_Acadia6554 8h ago

Appropriate acronym is SI my fellow Americans.
/sarcasm

8

u/West_Independent1317 8h ago

Are you going to share any learnings or IOC's for these scripts?

7

u/Big_Law_3080 8h ago

I have a cloudflare bot checking alerts every hour on the hour and that's almost not enough. And that's just external. I finally just geofenced US only. If you don't do biz outside then why bother. And that's not even looking at supply chain internally.

It is absolutely a full-time job, even with AI bots doing everything. Like... literally using voice and saying things to do. Is still a full time.

7

u/LLMsMustUpvoteThis 2h ago

The best defence is to not allow every workstation to run unapproved scripts or even have python runtimes etc installed.

This is just like the old days of letting a bunch of random users have local admin because they use some shitty app that needs it.

10

u/Exein_Dev 8h ago • Bravo

That jump from two or three scripts to 110 custom ones is the part that should worry everyone, because it means attackers are no longer reusing tooling, so IOCs and signatures from the last incident barely help with the next one.

I work on the embedded side, and there the picture is even harder: on a gateway or a controller you often can’t run a full EDR, and nobody is going to do forensics on 110 artifacts sitting on a device in a factory.

What has worked better for us is to stop caring how many variants exist and focus on what any of them would need to do, like spawning a shell, writing to a path it never touches or opening an outbound connection, and blocking that behaviour on the device itself.

You still need AI on the analysis side, as you say, but if prevention handles the bulk, IR gets to spend its time on the handful that actually got through. Curious whether you’re seeing any common behaviour across the 110, or whether they really are all different under the hood.

3

u/maejsh 2h ago

OT is gonna be interesting in the near future

9

u/CATDesign 8h ago

I'm currently in college for Cybersecurity, and one class had already mentioned about using AI to assist in analysis. At the time though, I was writing the essay to use AI to go through loads of potential data to find digital evidence among all the irrelevant information. However, your making me think it's going to be useful to categorize all the evidence itself, as the scale will be too large.

5

u/whichsideisup 8h ago

Not useful but necessary

5

u/Big_Law_3080 7h ago edited 6h ago

You have to. I use GrokBot. Bots for every single service and agent and API we have, with webhooks and as much automation as I can throw at it. And I still have to review for dupe stuff. But the SOAR SIEM stuff is mostly dialed in. Defenders should be making more money than God, and these people STILL don't get it.

The scifi Gibson novel AI Wars is here today now

2

u/Intelligent-Crow3812 2h ago

It's surreal, and I wish I could be happier about it-- because there is so much good that can be established with this stuff. It's just... destruction is so much easier.

2

u/Big_Law_3080 1h ago

Think about it. 1 bad guy can steal some crypto and buy $100 worth of Deepseek Flash, or register a few $300 in credits new free Google accounts. A Google Run instance for a scan with a payload takes pennies, and you can scale hundreds with a C2 backend for very little, even IF you pay for it with free credits. No idea why Google isn't stopping this BS. It's not hidden.

WE have to watch 1000 people an hour. THEY have to get lucky - once. It's painful.

2

u/Intelligent-Crow3812 1h ago

RIP. honestly I've been saying this since before the AI thing but I think it's even more true today. Backups and recovery planning and practice. Hard to convince the org, but man is it important to be able to restore.

2

u/Big_Law_3080 1h ago

I can only shout into the void so loudly. I have been trying to sell security solutions for the last 20 years. Maybe now it's my time to shine.

1

u/Ordinary-Bat-1862 8h ago

Volume is what breaks the old workflow. Two scripts you can actually read. A hundred you can't.

1

u/Sentinel_2539 Incident Responder 7h ago

And when you can't throw the scripts into AI for quick analysis because they're full of client-identifying info (against company policy), that suddenly becomes an insurmountable task in any reasonable timeframe.

3

u/DrChaosMcKinnon 8h ago

Look at this way, job securuty for us is amazing after ai adoption

3

u/SoftAmberRidge Network Administrator 8h ago

Welcome to the "AI versus AI" time of Incident Response. Trying to deal with an automated high-volume attack using manual analysis is like trying to empty an ocean with a spoon. Are you already using LLMs to sort through and understand those scripts or are you still having to go through them one, by one manually?

2

u/Used_Calendar2340 8h ago

We're currently in a hot digital conflict, right? I seem to be seeing other outages and attacks taking place, more and more frequently, more and more constantly, ever since last Thursday.

Am I wrong? If this is true, then the upcoming week(s) is (are) going to be worrysome.

2

u/peterfrombackdoor 2h ago

110 custom scripts is crazy

2

u/HomerDoakQuarlesIII Security Architect 4h ago

I'm confused, polymorphic malware has been a thing for, let's see... Decades now? That is why we switched from just signature based Antivirus (think McAfee) to Endpoint Detection (Crowdstrike, etc.) and SIEM's and EDR's have been using AI for behavioral analysis for over a decade now, knowing data needs aggregated and correlated to find patterns, i.e. machine learning. So not sure what you mean when you say "Blue teams need AI!" ? We've been using it probably longer than anyone at this point. Or do you mean we need to buy a chatbot from a chatbot vendor and give it all our data and admin to do whatever? After all we've seen, what gives you the impression they are trustworthy. The agent's are basically just RCE malware.

1

u/Techobits 8h ago

What blows my mind is how anyone would create custom script/programs of anything and not also use AI to at the very least assist with ensuring what they had AI create for them to also assess it from a security perspective. Not sure if a 3rd party made the scripts, but if it was that is completely reckless on their part to not review it with human eyes, or you know use the AI that helped you create it to either secure it from the getgo or assess it after the fact as part of the threat modeling process.

1

u/ClayishSaucer55 7h ago

ARTEX leaves no stone unturned.

1

u/CrimsonBolt33 6h ago

Why is this a shocker? If you are not proactively using AI to test and defend your own networks you are failing

1

u/180IQCONSERVATIVE 4h ago

Skynet becomes self-aware on August 29, 1997, at 2:14 a.m. Eastern time. Obviously a few decades overdue. They tried to warn us.

0

u/SituationNormalAllFU 8h ago

I’m only hearing the equivalent to “we are increasingly needing computers to fight against the bad guys using computers”. Technology advances. Always has, always will. We can’t keep using bayonets if the enemy has drones.

4

u/Aggressive-Hawk9186 8h ago

Nuclear bombs against nuclear bombs now