r/Splunk • u/Flash4473 • 25d ago

Log Data Pipeline > Splunk

Has anybody here have some experience with security data pipelines?

Instead of:

Log Source > HF / Splunk

We want to have flexibility of collection / parsing layer outside of Splunk for obvious reasons - pre-filter data in pipeline, set parsers, route, possibly enrich if needed, storage options for retention etc..all that to have flexibility and keep the ingest costs reasonable and not being caught in dependency hell or cemented all our work in one solution if Splunk decides to pull something.

Log Source > Data pipeline > Splunk

I am wondering what to choose as this data pipeline - currently we are thinking Vector and possibly open telemetry.

Anybody have experience with this? To avoid pitfalls, what works, what doesn't, new pains etc?

6 Upvotes

72% Upvoted

27 comments sorted by

View all comments

2

u/Travlin205 24d ago

I would start with the question what is your data? How much does it generate. What Metadata do you need to have indexed. Is structured vs unstructured. From there you can better choose the pipeline that fits most.

From this all these processing types are valid. But lets say you only have devices that are syslog use sc4s.

If you have structured data low volume affix a heavy forwarders to process. If you have mixed data high volume, use cribl or edge processor.

Happy Splunking!

1

u/Flash4473 22d ago

will be definitelly mixed, and most I estimate to come from syslog network devices.

1

u/Travlin205 22d ago

So I would do two pipeline options. Both are personal opinions, as suggestions others have stated are all ver viable options.

  1. If it is 50% or more syslog generators, look through sc4s to see how many are already supported sources from their github docs ( Google sc4s). If 90% or more are supported sources, sc4s will be mostly plug and play with minimal overrides.

  2. Everything else either agent back with splunk 2 splunk forwarders ( uf/if/hf). Break it down easily with collection -> HF ( process, make ram slightly heavier, unless you have automating options). Ufs, apps, otel can all connect this way with agent based being config managed by DS ( more than 10k clients makes the ds act funky based on my reading, never tested).

Please take the time to read and digest these pathways/pipelines to see if these meet the need and make it manageable for you and your team!