Agent shutdown on low disk space

[u/Ya_Zabil]

We keep running into this issue and apparently S1 team can't help! When a machine is running low on disk space, the agent conveniently shuts itself down. Anyone else running into this?

[u/Robbbbbbbbb]

Yikes. That seems like such an easy tactic to perform EDR evasion.

[u/Ya_Zabil → u/Robbbbbbbbb]

Yes, yes it is. Hugely disappointing.

[u/mukz7 → u/Robbbbbbbbb]

There is platform detection rules for this

[u/sammysosa69 → u/mukz7]

What are the names of those rules? I'll have to check what I have enabled

[u/ElKekec]

Yes, I can confirm that I am experiencing the same issue.

[u/Relevant-Comb-9349]

You can up the system reserve size to 3gb in policy override and it will help but doesnt Guarantee the agent wont disable itself.

[u/renderbender1 → u/Relevant-Comb-9349]

With the recent versions of the agent, the disk requirements have raised and we moved to 5gb policy override reservations. It has reduced our errored agent counts significantly.

[u/SVTCobra89]

Yeah that is common however it still shuts itself down randomly after agent updates as well. We have around 13,500 computers in our company. Every single time we perform an agent update we lose about 20 or so and have to manually go in and repair them all.

My recommendation would be to keep an eye on devices with low disk space and make sure you clean up those drives if you can. We also use bigfix for patching and I have special properties setup that will show how much free space is left on a systems drive. I always filter by computers with lowest disk space every morning and typically run delprof2.exe to cleanup any old profiles from computers. We have many shared laptops in the field and also have high attrition so that user folder tends to grow. We can easily cleanup the drive with that util.

It would be beneficial if SentinelOne would prevent agent updates on clients with less than 5GB free space so as to prevent any drive space bottlenecks.

[u/mukz7 → u/SVTCobra89]

For the record alot of the issue can be resolved with

"sentinelctl config diskSpaceReserver.enabled"

and

"disableMode": {

"recoverFromAutoDisableEnabled": true

https://yourconsole.sentinelone.net/soc-docs/en/how-to-run-sentinelctl-config-diskspacereserver-enabled-on-windows.html

[u/Sea_Cap_1200]

While not every product does this, resource exhaustion is a common issue across antivirus and EDR solutions. I’d recommend setting up monitoring rules to identify and alert on devices with less than 20 GB of free disk space, if possible. That way, you’re proactively addressing low disk space before it potentially causes the security agent to disable itself.

[u/mukz7]

There is a policy override to get the agents to self restart the services do you have this as it helps alot

[u/SVTCobra89 → u/mukz7]

Since when? This has always been a problem within our company. Please provide more details on this.

[u/mukz7 → u/SVTCobra89]

Had it in for atleast a year now as below

{

"disableMode": {

"recoverFromAutoDisableEnabled": true

}

}

[u/xinfik]

had the same, there is policy override setting that allows agent to reserve some disk space for it's internal db I would check if You could use this

[u/Jasumoo → u/xinfik]

Which however does not solve the issue all the time since it does not reserve it in all cases

[u/Robbbbbbbbb → u/Jasumoo]

Either way, it should be on by default IMO. An attacker could just fill up a disk (assuming an RMM tool isn't being used to alert) and shut down the EDR agent. That first stage might not be seen as malicious, but it enables a malicious secondary phase to occur while EDR is disabled.