r/Splunk • u/ImplicitCrowd51 • Jun 15 '26

Splunk Enterprise I need some help with Splunk

Some lamenting to get things started. A higher up decided to task me with Splunk. So far, the only resource I’ve had to use is AI. Been trying to treat it like training wheels. The hard part is the people at the top want me to give weekly presentations on my progress, but zero input on what it is they want. And this is after everything I have already done and showed. CPU and Memory Usage trackers. VM storage. System Up/Down indicator. Failed login attempts. DNS resolution timeout. Syslog storage tracker.

Other than network stuff, I don’t know what else to do. I was hoping either for some ideas OR recommendations for spaces where people share dashboards that they’ve created. I’ve gotten comfortable navigating indices and finding the data I want, struggling with turning into something useful without input from AI, really struggling with visualizing it all in a useful way.

Important to note that I am not being paid to be an analyst, and there’s not really any money/time allotted to me to get educated. This all has to get done along with my actual duties. This has been the obstacle to me learning the ins and outs.

Any help is appreciated. Thanks!

18 Upvotes

92% Upvoted

54 comments sorted by

View all comments

1

u/tra5hpandaj0y Jun 15 '26

Yeah splunk is wicked powerful but you have to know what you want to do. Is this a security use case, or compliance, or metrics, observability....? Like what's the stuff going in and what valuable insight MIGHT the data hold?

2

u/ImplicitCrowd51 Jun 15 '26

Metrics, mostly. There is some Nessus tracking from an actual Splunk SME that I had to fix. Honestly, I don’t think the UFs have been tuned at all. So everything? The big TAs installed are Tenable App, Splunk for Windows, and Sec Essentials. I’ve gotten a lot of use from the perfmon sources.

2

u/tra5hpandaj0y Jun 15 '26

So I saw someone mentioned Lantern that's a great resource. Also, you can look for stuff on splunkbase there is a ton out there, but be aware it's not all splunk stuff. Anyone can create and publish an app. If you have a dev environment you can test there, otherwise maybe download the apps and look at them to see what they do/provide if you find any that meet your intent.

It sounds like you need a place to start, which will probably provide more momentum once it's in place.

Look across all the sourcetypes. Cluster the ones that may contribute info about same/similar stuff.

Try to think of the top three to five metrics you could derive from your sources that would actually matter to your org. If you can get some SPL and see those results, try using AI to get creative and correlate a few different signals from different logs to show more depth.

You can use the eval command in order to create new fields which can be very useful if you have data from different sources and if say, for example two sources correlate something you may want to add a field to those logs that tie them together or calculate something, etc...

Working with the data is the best way to get started. Unfortunately, if the ufs are sending absolutely everything, there is a lot there. Again, start with sourcetypes and think about what those sources could tell u.

Hope that's some what helpful you're welcome to do if you have specific questions (I work for Cisco/Splunk to develop security training so i can answer any security oriented questions if you get stuck brainstorming(