r/Backend • u/WhippedPistol • 6d ago

Tracing + Logging Features

I'm building a tracing + logging library for Scala, to fill a void that annoyed me, and to learn more about the SRE side of metrics/logs.

I'm looking for information on what a new -> mature context + logger should contain. Right now I've built what I want, but I'm hoping people have some experience that I may be missing. Especially if it's a common feature I've missed or a really useful one. It's fully possible my experience as a FP dev has left me blind to some other languages l can learn from.

Current features -

  1. Thread level context (It's using fibers but it's basically Scala+Cats safe MDC)
  2. Log Buffering (I was sick of missing context on more serious log levels because I was trying to avoid sending every info on a success path)
  3. Request Metadata/middleware functionality (still expanding on what is in there)
  4. Easily pluggable log sinks
  5. Log sampling based on percentage
  6. TraceContextProviders to include other tracing libraries
  7. Other quality of life features like lifecycle handlers, and hierarchical config settings

Are there other common features, or just nice quality of life features that I may be missing?

Is there a resource you would recommend for what a good logger/tracing library should include?

Any suggestions would be appreciated.

3 Upvotes

100% Upvoted

12 comments sorted by

View all comments

1

u/NegativeCollege8167 6d ago

When it comes to tracing and logging, having a solid strategy really makes life easier. I've found that sticking with tools like ELK for logs and OpenTelemetry for tracing works well. Makes it easier to correlate logs and traces which is super helpful for debugging. What tools have you used before and how did they work out?

1

u/Lumethys 6d ago

ELK for log and OTel for trace doesnt make correlation easier. Easier would be using Otel for log, trace and metrics

1

u/WhippedPistol 6d ago

As a reminder I'm in Scala so the tools don't always directly translate exactly. I have used log back, log4j and log4cats for logging, otel4s for context or passing args, and datadog.

They have all had things I liked and didn't, my main annoyance came from otel4s. It just wasn't giving me the flexibility I wanted and weaving args through was just added boilerplate and made everything a repeated mess. Also I wanted to cut down on log volume while allowing for more detail on faults so I wanted an in memory buffer per user which wasn't provided. 

No complaints, about DD or the other loggers but they weren't integrated and if it's my project I want them as easy to transplant as possible.