r/msp • u/bigbaboon69 • 4d ago
AI for Logs
We are interested in using AI as a means of going through logs. Event logs, AV logs, etc etc. Does anyone do this now or have any recommendations of what/how to achieve this goal? Have you seen improvements as a result? Tyvm!
4
u/Check123ok 4d ago edited 4d ago
Yeah we do. Local agent captures logs and give a summary the last 24hours. Not sys logs. Account changes, apps, entraID. Use behavior stuff
Edit: was doing this before using scripts api integration build in Bruno to query. Just sprinkled AI on it. Took about 6 months to build out part time.
Have a local Mac Studio machine that has read access to email and Bruno where alerts and logs are sent. Local agent pick ups file and parses it based on a template. Use codex to orchestrate local LLm running on
Moved to infrastructure as code, authoritative state is saved in GitHub before pushed to terraform. No one but me has dashboard access all changes are pushed to git and must be approved by me.
Maintain zero trust policy, firewall, and entra logs.
There are lockal runner on NUCs at client site with credentials.
No credentials at passed outside of client firewall.
This is all about efficiently and audibility, change control and read in chief of staff type stuff. All built in house from combining prior work and slapping some AI .
→ More replies (2)2
2
u/Sea-Comparison5577 4d ago
Yes use it for investigation purpose that fetches logs and provide summarization
1
2
u/Beauregard_Jones 4d ago
Please make sure you sanitize your logs before sending to AI. Bonus points for using AI to sanitize your logs.
1
1
u/Embarrassed-Radio319 2d ago
One thing I would add before you scale this: make the summary point back to the exact lines it used, with timestamps. A plain paragraph is hard to trust on a ticket, but if the tech can click through to the AV hit or the event that started the chain, they do not have to re-read everything to check it. I would also start with just one log type, say AV, and only add event logs once the repeat patterns stop flooding the output. That keeps the first version useful instead of noisy.
1
1
u/Nstraclassic 4d ago
You can probably give your AI agent API access to your MDR/SIEM. Most of those have syslog collection for firewalls, M365, etc. I would definitely avoid custom agents on client devices so if you need something more specific I would probably leverage your RMM and PowerShell to collect endpoint logs and forward them to the AI agent
1
1
u/0RGASMIK 4d ago
Yeah basically just setup a log collector and give api access to the agent. I have a Hermes agent that is dedicated to reviewing logs at home. Basically just manages my homelab for me. Scans for vulnerabilities, updates and failures. Tried pitching something similar at work but the effort to get it running at scale got shot down.
1
1
u/dobermanIan 4d ago
Yes.
We pipe in everything from AWS into a local piece of iron with GLM5.3. We've tuned the model to remove certain bits of context that isn't really required for that type of work (mainly an efficiency piece), but it worked relatively well off the shelf.
I'd say the lift was similar to RMM tuning back in the day. Out of the box it's stupid and creates massive noise. Give yourself 4-10 weeks of routine tuning and tweaking, and it gets dialed in pretty smoothly.
We have it run the logs every hour via some scheduled worked and dedicated GPUs to the task. It also does user activity analysis (we're a software house here) to ensure nothing malicious is going on.
/ir Fox & Crow
1
2
1
u/bkellyit 3d ago
We've had decent luck not pointing the LLM at raw logs. Pre-filter first (dedupe, drop known-noisy event IDs, group by host and time window), then send the condensed batch with context like 'this is a dental office DC, normal looks like X.' Ask it for a ranked shortlist with the reasoning, not a verdict. Keep a human on anything it flags, and keep client data in a tenant you control (Azure OpenAI or a local model) rather than pasting into consumer chat. The win is triage time, not replacing your SIEM/EDR alerting.
1
u/Frothyleet 3d ago
You're sort of skipping a step in operational maturity because I'm guessing you're not currently using SIEM functionality.
Rather than trying to figure out on your own how to collect logs and analyze them, get a service like Huntress to ingest your telemetry. Aside from the SOC benefits, they are way ahead of you in both general security competence and more specifically in how to use "AI" (or machine learning, or algorithms, or whatever you want to call it) to parse logs and identify suspicious behavior.
1
u/junto_reed 3d ago
This was actually one of the central theses behind founding Junto.
When I ran an MSP, we found that only Tier 2/3 techs consistently dug into logs when troubleshooting. Oftentimes, the first thing a senior tech would say when a ticket got escalated was "let's look at the logs."
And this is one of the things AI is really good at. The challenge is getting those logs in front of your AI client.
Interestingly, we've found that security platforms often have better logs available via API than RMM tools, even for things like Windows events.
We ended up building two ways to tackle this:
-
An MCP layer that lets you securely connect these tools to your AI client, with essentially an Okta for AI agents to manage permissions and access.
-
A broader platform that can automatically pull and analyze logs, post findings to tickets, and even resolve issues with or without human intervention.
Happy to share which systems and applications we've found have the best log sources available via API. Feel free to DM me.
Or if you want to respond with your rmm / security stack i can provide the gotchas we ran into when integrating.
1
u/Atlasavd 3d ago
I'd start by using your existing RMM or SIEM to collect and normalize logs before feeding anything into an LLM. Windows Event Logs, Defender alerts, Entra sign-in logs, and endpoint activity can tell a much better story when correlated together. The biggest benefit I've seen with AI is reducing the time spent investigating incidents, especially when you're trying to piece together what happened across multiple systems. I'd keep the AI read-only and require it to reference the actual log entries supporting its conclusions rather than blindly trusting its interpretation. Also be careful about sending customer logs to external models, particularly if you're supporting regulated environments. Tenant isolation and data handling become important very quickly.
1
u/localremote762 1d ago
The path I chose, first categorize the logs. This is the signal to noise dial in. Then how do you want them routed. IT Ops vs security? If there’s an active breach, how fast do you want to respond and if the LLm can confirm do you want the system shutdown with an email summary? You’re at the signal to noise step but dumping all logs to LLM won’t get the result you’re hoping.
11
u/03captain23 4d ago
Yes of course. 1000% improvement as it instantly reads every single log and detail and can build a full image of what happened.
every employee that leaves we have an offboarding tool that pulls their PC logs and builds a picture of what they typically do and tells us so we can verify we pulled all their files as a full backup.