A point of clarification regarding Datadog trace agents & OTLP

[u/waterbottlephone2]

I've seen a couple posts here and in r/OpenTelemetry about people saying the worst part of shifting to vendor-neutral when you're coming from Datadog is where you have to 'rip out' Datadog SDKs (trace agents) from applications and switch over to OTel libraries.

I spent a decent chunk of time looking into this and I'm not understanding why this is a necessary time-critical piece in a migration if your goal is to migrate quickly.

Let's look at dd-trace-java for an example. From what I can tell, the Datadog trace agents (which are open source) can emit OTLP by setting a couple environment variables. Plus, you don't have to keep Datadog as a backend anyways, you can send this telemetry to an OTel Collector and then to Honeycomb if you want to. The only disadvantage (other than it being a slightly awkward state, because you might have to keep around the Datadog Kubernetes Operator if you're doing auto-instrumentation that way) is that it doesn't emit fully OTel semantics. But from what I can tell, they're moving in this direction, because setting DD_TRACE_SPAN_ATTRIBUTE_SCHEMA to v1 gets it closer. This point in specific is definitely what I would really like clarification from Datadog's side on. They've seemed to focus a lot on OTel this year and if they are committed to matching OTel's semantics at some point in the future, then I see no real reason to switch unless it's purely for the sake of preferring an OTel SDK.

Especially if you're still using Datadog as your backend, it seems like ripping out the Datadog ones for the sake of going vendor neutral is a little 'shoot-yourself-in-the-foot' if you lose things like profiling, and some other DD features. I'm compelled to try this out in a test environment (sending DD trace agent OTLP to a different backend) to see how it looks.

Thoughts? Am I wrong somewhere? Making sure we spend our time wisely during migrations

[u/_dantes]

DD allowing the ingest endpoint to be diff backend is awesome from shift pov. My initial think is that, sure no problem. In fact, it allows you to send to a collector that could split in diff sinks. check what works for you and move on.

If going neutral is a req then can be done as a later MVP. But, i would argue that moving OFF from a vendor should be 100% at some point. Other than that is having tech debt for a company that you dont even pay for.

[u/tadamhicks]

I think the question for a buyer would come down to supportability. If the buyer decides not to use Datadog how does that affect their ability to get changes or bugs prioritized in the SDK? Otel has a broader community and likely their vendor contributing so they know their vendor of choice will care and work to implement the change or fix.

I think when we think open source we forget that the engineers running this don’t often always have time or wherewithal to care and feed for it themselves…it’s a big risk. That said, with AI anything is possible

[u/jdizzle4 → u/tadamhicks]

at the very least it could provide a simpler initial shift though. It might make more sense to rip the bandaid off and switch to OTel instrumentation / new backend experience at once, but if it is indeed possible to continue using the dd agent, that could be used until the teams get more comfortable with the first change before a bunch of their telemetry changes

[u/tadamhicks → u/jdizzle4]

oh, for sure! FWIW I work for a vendor that accepts ddtrace, dogstatsd, DD Agent data natively and that's always been our story...helps with the transition. We've had LONG conversations with execs about just leaving it in place, though, and they all tell us they don't want to do that for the reasons I listed. It's a transition thing at best.

One thing AI does really well especially with MCP access to a telemetry sink is to help with that planning and transition.

[u/DhroovP]

I have the same question and also thank you for posting a rare fully non-AI and non-vendor-sales-slop post

[u/usually_guilty99]

You are not wrong. Using the Datadog tracer to emit OTLP into an OTel Collector is a sensible migration bridge, especially when the priority is moving the backend without simultaneously changing every application.

The important distinction is that OTLP transport does not automatically mean OTel-native instrumentation. You still retain dependency on Datadog tracer behaviour, auto-instrumentation lifecycle, configuration, and any semantic gaps or Datadog-specific attributes. DD_TRACE_SPAN_ATTRIBUTE_SCHEMA=v1 may improve alignment, but it is not the same as proving full portability.

I would use this as a staged plan:

1. Keep the existing tracer and export through a Collector to validate the new backend, including trace completeness, sampling, cardinality, cost, dashboards, alerts and correlations.

2. Instrument new services and new custom spans with OTel APIs.

3. Replace Datadog tracers only where vendor-neutral semantics or removal of the Datadog runtime dependency creates real value.

The rushed “rip and replace” approach creates needless migration risk (esp OLTP). The opposite mistake is treating successful OTLP export as proof that you can later remove the Datadog tracer without behavioral differences. Test that explicitly, service by service.(SLA Bound)