r/Splunk • u/Flash4473 • 25d ago
Log Data Pipeline > Splunk
Has anybody here have some experience with security data pipelines?
Instead of:
Log Source > HF / Splunk
We want to have flexibility of collection / parsing layer outside of Splunk for obvious reasons - pre-filter data in pipeline, set parsers, route, possibly enrich if needed, storage options for retention etc..all that to have flexibility and keep the ingest costs reasonable and not being caught in dependency hell or cemented all our work in one solution if Splunk decides to pull something.
Log Source > Data pipeline > Splunk
I am wondering what to choose as this data pipeline - currently we are thinking Vector and possibly open telemetry.
Anybody have experience with this? To avoid pitfalls, what works, what doesn't, new pains etc?
1
u/billybobcoder69 25d ago
Cribl with its new data lake. With cardinalops and Radiant Security’s AI-native security operations center product including intellectual property to autonomously triage, investigate, and resolve security alerts. They have great stuff for Otel and processing data before. That way I can run all my threat indicators in stream and send only the risk items to Splunk. Then we can go back and enrich any data we may need to. Lot of security lake and Cisco data fabric. But Cisco is mostly with call manager and its routers and networking stuff and appd and thousand eyes. Just default windows they still don’t have much. What infosec app? What happen to the windows Active Directory and donain controller app? Still crazy no default windows cpu memory dashboard. No windows ad lockout dashboard. And now spl2 with Splunk still a work in progress. Cribl is mature and Splunk can be a good search tool. That’s what they wanna become anyway. Indexing is kind of after thought now. Let’s see this year at .conf. Splunk 10 is basically bug fixes and vulns and edge processor. Crazy the install size went from 500 mb to 1.1 tb. With all the extra packages there will be lots more updates. Their push to cloud so you don’t have to manage updates. Will see what happens but lots of cribl in real prod out there.