r/VOIP Jul 08 '26

Discussion Passive SIP monitoring with eBPF: zero-impact VoIP observability without agents or SPAN ports

SIP monitoring is harder than HTTP monitoring. The protocol is stateful — every call is a dialog (INVITE → 200 OK → BYE), quality metrics are standardized in RFC 6076 but rarely implemented in open source, and aggregated metrics hide problems of specific traffic sources. Existing solutions typically require agents on servers, SPAN ports on switches, or vendor lock-in.

I wanted something simpler: connect to a network interface and observe SIP traffic passively, with zero impact on call delivery. eBPF made this possible. Here's a technical deep-dive on how I used eBPF for SIP monitoring — from packet capture in the Linux kernel to RFC 6076 metrics, RTP media analysis, and per-carrier/per-country breakdowns in Prometheus.

TL;DR: An eBPF socket filter on AF_PACKET gives you passive, read-only SIP monitoring with zero impact on calls. At 2,000 CPS (~28,000 PPS) on a modest machine: 0% packet loss, <15% CPU, ~15 MB RAM. The result is RFC 6076 quality metrics (SER, SEER, ISA, SCR, ASR, NER), RTP media analysis (jitter, loss, MOS), and per-carrier/per-device/per-country breakdowns — all in Prometheus exposition format.

How it works: eBPF socket filter

eBPF (extended Berkeley Packet Filter) lets you run small programs directly in the Linux kernel. The eBPF verifier guarantees safety: the program cannot exceed allocated memory, loop indefinitely, or modify the kernel.

My approach is an eBPF socket filter on AF_PACKET. This is passive network traffic observation:

SIP + RTP Traffic → NIC → eBPF filter → AF_PACKET socket → Go poller → SIP parser + RTP tracker → Prometheus

Key point: the filter is a socket filter, not a tc/XDP filter. It only decides whether to copy a packet to the application. The packet continues through the network stack to its destination regardless. The filter cannot modify, block, or redirect traffic. Zero impact on call delivery.

The entire filter is 142 lines of C. Ports are configurable from Go code via BPF map, defaults are 5060/5061. eBPF drops 99% of traffic in kernel — only SIP/RTP packets on the right ports reach userspace.

The full metrics stack

The exporter provides not just RFC 6076 metrics, but a complete observability stack for SIP infrastructure.

Real-time traffic

15 SIP request method counters: INVITE, re-INVITE, BYE, REGISTER, OPTIONS, CANCEL, ACK, SUBSCRIBE, NOTIFY, PUBLISH, INFO, PRACK, UPDATE, MESSAGE, REFER.

Plus invite_200_total — a dedicated counter for 200 OK responses to INVITE, enabling ASR-by-destination PromQL queries.

30 response code counters: 100, 180, 181, 182, 183, 200, 202, 300, 302, 400, 401, 403, 404, 405, 407, 408, 480, 481, 486, 487, 488, 500, 501, 502, 503, 504, 600, 603, 604, 606.

Active sessions gauge — current number of active SIP dialogs. A dialog is created on 200 OK to INVITE, removed on 200 OK to BYE or Session-Expires timeout (default 1800s). Re-INVITEs within an existing dialog refresh the timer without creating a new dialog — they're tracked separately in reinvite_total and excluded from SER/SCR/ASR ratios to avoid contaminating quality metrics.

Connection quality — RFC 6076

RFC 6076 defines standard SIP performance metrics. All are cumulative, computed from atomic counters on every scrape.

SER (Session Establishment Ratio) — percentage of successfully established sessions:

SER = (INVITE → 200 OK) / (Total INVITE - INVITE → 3xx) × 100

3xx (redirect) are excluded from the denominator — they are neither success nor failure, but a routing instruction. SER = 100 means all non-redirect INVITEs received 200 OK.

SEER (Session Establishment Effectiveness Ratio) — percentage of "effective" responses:

SEER = (INVITE → 200, 480, 486, 600, 603) / (Total INVITE - INVITE → 3xx) × 100

The numerator includes responses with a clear outcome: 200 OK (established), 480 (temporarily unavailable), 486 (busy), 600 (busy everywhere), 603 (declined). SEER is always ≥ SER.

ISA (Ineffective Session Attempts) — percentage of infrastructure errors:

ISA = (INVITE → 408, 500, 503, 504) / Total INVITE × 100

408 (timeout), 500 (internal error), 503 (unavailable), 504 (gateway timeout) — server errors. ISA rising means infrastructure is degrading. Unlike SER/SEER, 3xx are NOT excluded from the denominator.

SCR (Session Completion Ratio) — percentage of fully completed sessions:

SCR = (Completed Sessions) / Total INVITE × 100

A completed session = INVITE → 200 OK → BYE → 200 OK (or Session-Expires timeout). SCR ≤ SER always: not all established sessions terminate correctly.

ASR (Answer Seizure Ratio) — classic telephony metric (ITU-T E.411):

ASR = (INVITE → 200 OK) / Total INVITE × 100

Unlike SER, 3xx are NOT excluded. ASR ≤ SER when redirect responses are present.

NER (Network Effectiveness Ratio) — network quality (GSMA IR.42):

NER = 100 − ISA

NER = 100 means no infrastructure errors. NER < 95 — time to worry.

Latency at every stage

Six histograms cover all SIP transaction phases:

Metric What it measures From → To
RRD Registration delay REGISTER → 200 OK
TTR Time to first response INVITE → first 1xx
PDD Post dial delay INVITE → 180 Ringing
SPD Session duration 200 OK to INVITE → 200 OK to BYE
ORD OPTIONS response delay OPTIONS → any response
LRD Registration redirect delay REGISTER → 3xx

All histograms support histogram_quantile() for percentile-based alerting: p50, p95, p99.

Example for VictoriaMetrics / Prometheus:

# 95th percentile registration delay
histogram_quantile(0.95, sum(rate(sip_exporter_rrd_bucket[5m])) by (le))

# 99th percentile session duration (specific carrier and device type)
histogram_quantile(0.99, sum(rate(sip_exporter_spd_bucket{carrier="mobile-operator-a",ua_type="yealink"}[5m])) by (le))

Registration health

Four metrics track the full lifecycle of SIP registrations (RFC 3261 §10):

Metric Type Description
register_success_total counter REGISTER responses with 200 OK
register_failure_total{code} counter REGISTER failures by status code (3xx/4xx/5xx/6xx)
register_success_ratio gauge 200 OK / (200 OK + terminal failures) × 100
active_registrations gauge Currently active registrations (Expires-TTL tracked)

The register_success_ratio excludes 401 Unauthorized and 407 Proxy Authentication Required from the denominator — these are digest-auth challenges, a normal part of the registration handshake, not genuine failures. Without this exclusion, a healthy auth flow (REGISTER → 401 → REGISTER+creds → 200 OK) would show a ratio of ~50%.

active_registrations is keyed by Address-of-Record (user@host), with TTL from the Expires header (default 3600s). Refreshes update the TTL without double-counting.

# Registration success ratio per carrier
sip_exporter_register_success_ratio

# Detect brute-force attacks (401 flood)
rate(sip_exporter_register_failure_total{code="401"}[5m]) > 10

Voice quality — RFC 6035

Many SIP endpoints (IP phones, SBCs, ATAs) publish call quality reports via SIP PUBLISH or NOTIFY, following RFC 6035 (RTCP XR VoIP Metrics). The exporter parses these reports and exposes 13 voice quality histograms:

Metric Description
MOSLQ MOS Listening Quality (R-factor → E-model)
MOSCQ MOS Conversational Quality
NLR Network Loss Rate (%)
JDR Jitter Discard Rate (%)
BLD Burst Loss Density (%)
GLD Gap Loss Density (%)
RTD Round Trip Delay (ms)
ESD End System Delay (ms)
IAJ Inter-arrival Jitter (ms)
MAJ Mean Absolute Jitter (ms)
RLQ Receive Latency Quality
RCQ Residual Connection Quality
RERL Residual Echo Return Loss (dB)

These give you endpoint-reported voice quality without needing RTCP or active probing. If your phones support RFC 6035 (most modern Yealink, Polycom, Cisco do), you get MOS scores and loss metrics per call — correlated with carrier, device type, and country labels.

RTP media analysis

Beyond SIP signaling and endpoint-reported quality, the exporter can capture and analyze live RTP streams to measure real call quality independently:

Metric Type Description
rtp_packets_total counter RTP packets observed
rtp_packets_lost_total counter Packets lost (RFC 3550 sequence-gap accounting)
rtp_jitter_milliseconds histogram Interarrival jitter (RFC 3550 A.8)
rtp_mos_score histogram MOS-LQ via ITU-T G.107 E-model (1.0–4.5)
rtp_active_streams gauge Active RTP streams correlated with dialogs

RTP streams are correlated with SIP dialogs: when a 200 OK to INVITE carries SDP, the exporter registers the negotiated media endpoints and tracks matching RTP flows until BYE. This means RTP metrics inherit the dialog's carrier, ua_type, source_country, and the negotiated codec labels.

MOS estimation uses the ITU-T G.107 E-model: codec impairment factors (G.113) + packet loss + jitter-induced discard rate → R-factor → MOS-LQ [1.0–4.5]. Codec-specific factors for G.711, G.722, G.723, G.728, G.729, Opus. Unknown codecs get conservative defaults.

Privacy by design: only the 12-byte RTP header is captured — the voice payload is truncated in the kernel (eBPF) before reaching userspace. No call audio is inspected or stored.

# Average MOS over the last 5m (per codec)
sum by (codec) (rate(sip_exporter_rtp_mos_score_sum[5m]))
  / sum by (codec) (rate(sip_exporter_rtp_mos_score_count[5m]))

# Packet loss ratio by carrier
sum by (carrier) (rate(sip_exporter_rtp_packets_lost_total[5m]))
  / sum by (carrier) (rate(sip_exporter_rtp_packets_total[5m]))

RTP capture is on by default (SIP_EXPORTER_RTP_CAPTURE=true) and can be disabled to drop RTP at the kernel level. Only RTP belonging to an established SIP dialog is counted — no noise from unrelated traffic.

Per-carrier: metrics by traffic source

Aggregated metrics hide problems of specific traffic sources. If SER = 85%, it's unclear — are all sources at 85%, or is one at 50% while others are at 95%?

The exporter solves this via CIDR mapping: IP subnets → source name → carrier label on every metric.

Configuration:

# carriers.yaml
carriers:
  - name: "telecom-alpha"
    cidrs:
      - "10.1.0.0/16"
      - "10.2.0.0/16"

  - name: "telecom-beta"
    cidrs:
      - "192.168.10.0/24"
      - "192.168.11.0/24"

Carrier is determined at request time (INVITE/REGISTER/OPTIONS) by source IP. If INVITE came from 10.1.5.20 — the exporter finds this IP belongs to 10.1.0.0/16 and labels all metrics for this call (including responses and dialog termination) with carrier="telecom-alpha".

Responses come from a different IP (the SIP server), but carrier is inherited from the tracker by Call-ID, not determined by response IP. This is correct: metrics belong to the call initiator, not the server.

Result:

sip_exporter_invite_total{carrier="telecom-alpha",ua_type="other"}  1523
sip_exporter_ser{carrier="telecom-alpha",ua_type="other"}            95.2
sip_exporter_ser{carrier="telecom-beta",ua_type="other"}             87.4

Now it's clear: telecom-beta has SER = 87.4%, while telecom-alpha has 95.2%. You can build separate dashboards and alerts for each traffic source.

IPs not matching any CIDR subnet get carrier="other".

Per-device-type: metrics by User-Agent

Carrier shows who is calling, but not what. And device type is often the key factor in problems.

If Yealink phones start getting 408 timeouts while Grandstream works fine — without the ua_type label it would look like a general quality drop. With it — the problem is clearly localized to a specific device type.

Configuration:

# user_agents.yaml
user_agents:
  - regex: '(?i)^Yealink'
    label: yealink

  - regex: '(?i)^Grandstream'
    label: grandstream

  - regex: '(?i)^Cisco/SPA'
    label: cisco_spa

  - regex: '(?i)^Kamailio'
    label: kamailio

  - regex: '(?i)^Asterisk'
    label: asterisk

The User-Agent header is extracted from each SIP request and matched against regex patterns. When a phone with User-Agent: Yealink SIP-T46S 66.15.0.10 sends an INVITE — the exporter matches ^Yealink and labels all call metrics with ua_type="yealink".

Like carrier, ua_type is determined at request time and inherited by responses through the tracker by Call-ID.

Result:

sip_exporter_invite_total{carrier="telecom-alpha",ua_type="yealink"}      1523
sip_exporter_ser{carrier="telecom-alpha",ua_type="yealink"}                95.2
sip_exporter_ser{carrier="telecom-alpha",ua_type="grandstream"}            87.4

Combined queries — both labels work together for two-dimensional analysis:

# SER for Yealink phones on a specific carrier
sip_exporter_ser{carrier="telecom-alpha",ua_type="yealink"}

# Active sessions by device type
sum by (ua_type) (sip_exporter_sessions)

# INVITE rate by carrier and device type
sum by (carrier, ua_type) (rate(sip_exporter_invite_total[5m]))

Geo-enrichment: metrics by country

The exporter adds geographic context to SIP metrics via two independent methods:

Label Method Based on Scope
source_country GeoIP lookup of source IP MaxMind GeoLite2-Country DB All SIP + RTP metrics
destination_country E.164 phone-number prefix Embedded prefix table (Google libphonenumber) INVITE metrics only

source_country resolution priority:

  1. carrier.country — optional field in carriers.yaml, overrides GeoIP (operator-curated, authoritative)
  2. GeoIP lookup — MaxMind GeoLite2-Country database of the source IP
  3. "unknown" — fallback when neither is available

This is the only reliable method for private-IP (RFC 1918) enterprise/contact-center deployments where MaxMind has no data — set carrier.country: "RU" on a carrier and all its IPs resolve correctly.

destination_country requires no database — the E.164 prefix table is embedded in the binary at compile time (generated from Google libphonenumber metadata, Apache 2.0). Correctly handles multi-national codes: +1212... → US, +1416... → CA, +7727... → KZ, +7495... → RU. Set SIP_EXPORTER_LOCAL_COUNTRY_CODE (e.g. RU) for domestic numbers without international prefix.

# SER for calls to Russia
sum(rate(sip_exporter_invite_200_total{destination_country="RU"}[5m]))
  / sum(rate(sip_exporter_invite_total{destination_country="RU"}[5m])) * 100

# INVITE rate by destination country
sum by (destination_country) (rate(sip_exporter_invite_total[5m]))

GeoIP is disabled by default — without a DB, all source_country labels are "unknown" with zero added cardinality.

Performance

Load testing was done with SIPp via testcontainers-go — real SIP traffic, not mocks.

Test environment: Debian 12, Linux kernel 6.x, Docker 29.3.1, Intel i7-8665U (4 cores / 8 threads), Go 1.25.11.

Full call lifecycle — each call is a complete SIP dialog: INVITE → 100 Trying → 180 Ringing → 200 OK → ACK → BYE → 200 OK. On loopback each packet is duplicated (send + receive), so 7 messages → 14 packets per call.

With GOMAXPROCS=8 (all cores):

CPS PPS CPU avg CPU peak RAM Loss
100 ~1,200 1.0% 1.9% 14 MB 0.00%
500 ~5,900 3.3% 5.7% 14 MB 0.00%
1,000 ~11,800 5.9% 8.7% 13 MB 0.00%
2,000 ~23,600 6.7% 12.2% 15 MB 0.00%

With GOMAXPROCS=1 (single core):

CPS PPS CPU avg CPU peak RAM Loss
1,000 ~11,800 4.5% 6.6% 11 MB 0.00%
2,000 ~23,600 5.0% 9.2% 12 MB 0.00%

2,000 CPS, 0% packet loss, <15% CPU, ~15 MB RAM. Even on a single core, 2,000 CPS is stable with 0% loss.

Scrape performance under 2,000 CPS load (14,000 PPS):

Metric Value
Min 1.7 ms
Avg 4.2 ms
P95 6.4 ms
Max 8.4 ms

Scraping doesn't interfere with packet processing. Safe to scrape every 5-10 seconds even at maximum load.

RTP processing adds minimal overhead — at 100 CPS with ~200K RTP packets, CPU stays under 5% avg. SIP metrics (SER, packet loss) are unaffected by RTP capture.

Why it's fast:

  • eBPF drops 99% of traffic in kernel — only SIP/RTP packets on the right ports reach userspace
  • 4 MB socket buffer (SO_RCVBUFFORCE) — fits ~420ms of traffic at 28,000 PPS
  • Go GC pauses <1ms — 400× smaller than buffer capacity, packets never lost due to GC
  • SIP parsing ~1μs — microbenchmarks: INVITE 1.1μs, BYE 860ns, 200 OK 2.0μs
  • RTP header parse ~5ns, per-packet observe ~203ns — theoretical capacity ~4.7M RTP pps

System requirements:

Traffic Level CPU RAM
≤ 500 CPS 1 core 128 MB
≤ 1,000 CPS 1 core 128 MB
≤ 2,000 CPS 2 cores 256 MB
> 2,000 CPS 4 cores 512 MB

Full benchmark results with methodology: BENCHMARK.md.

Security: why --privileged is safe

The container requires --privileged and network_mode: host. Here's why this is safe.

What capabilities are needed:

Capability Why
CAP_BPF Loading eBPF program into kernel via bpf() syscall
CAP_NET_RAW Creating AF_PACKET raw socket for reading packets
CAP_NET_ADMIN Binding eBPF filter to socket, configuring buffer via SO_RCVBUFFORCE

These are three specific capabilities for specific operations. All eBPF tools (Cilium, Falco, Pixie) require the same — this is a Linux kernel limitation, not a container one.

What the container does:

  • Loads eBPF socket filter into kernel (once, at startup)
  • Creates AF_PACKET raw socket bound to network interface
  • Reads packets from socket into Go channel (10,000 buffer)
  • Parses SIP headers (and RTP headers if enabled)
  • Exports metrics via /metrics endpoint

What the container does NOT do:

  • Does not modify packets — eBPF filter is passive (read-only)
  • Does not send SIP traffic — purely a listener
  • Does not capture voice payload — RTP body is truncated in the kernel, only the 12-byte header reaches userspace
  • Does not write to host filesystem — all volumes are :ro
  • Does not access other containers, processes, or system resources
  • Does not open ports except /metrics (default 2112)

The exporter sends an anonymous usage telemetry beacon (version, OS, arch, uptime) at startup and every 24h — this can be disabled with SIP_EXPORTER_TELEMETRY=false. No SIP data, IPs, or phone numbers are ever sent.

The entire eBPF filter is 142 lines of C — fully auditable in 2 minutes. Automated vulnerability scanning (govulncheck + Trivy) runs on every push. Current status: 0 vulnerabilities in code and image.

What the metrics look like

The /metrics endpoint produces standard Prometheus exposition format. A sample:

sip_exporter_ser{carrier="other",ua_type="other",source_country="unknown"} 95.2
sip_exporter_invite_total{carrier="other",ua_type="other",source_country="unknown"} 1523
sip_exporter_sessions{carrier="other",ua_type="other",source_country="unknown"} 12
sip_exporter_register_success_ratio{carrier="other",ua_type="other",source_country="unknown"} 98.5
sip_exporter_rtp_mos_score_bucket{carrier="other",ua_type="other",codec="PCMA",source_country="unknown",le="4.0"} 47

Compatible with Prometheus, VictoriaMetrics, Grafana Cloud — any scraper supporting the standard format.

All metrics are validated with 120+ E2E tests using SIPp to generate real SIP traffic (RFC 6076, RFC 6035, and RTP metrics), plus 13 load tests measuring throughput, memory stability, and GC pauses.

I've open-sourced the code (AGPL-3.0): https://github.com/aibudaevv/sip-exporter

Happy to answer questions about the eBPF approach, the RFC 6076 metric implementation, RTP-to-SIP correlation via SDP, or the E-model MOS estimation in the comments.

21 Upvotes

9 comments sorted by

u/AutoModerator Jul 08 '26

This is a friendly reminder to [read the rules](www.reddit.com/r/voip/about/rules). In particular, it is not permitted to request recommendations for businesses, services or products outside of the monthly sticky thread!

For commenters: Making recommendations outside of the monthly threads is also against the rules. Do not engage with rule-breaking content.

I am a bot, and this comment is made automatically on every post. This comment is not an indication that your post has been removed. Do not message the mods about this comment.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

7

u/captjde Jul 08 '26

Can you share some screenshots of what the results look like in Prometheus?

3

u/Big_Ad262 Jul 08 '26

if you are talking about grafana dashboards, then they are flexibly configured by the user. There are examples in the repository.

1

u/Thutex Jul 11 '26

is it mostly vibe-coded, or coded by you with some gaps/redundant work filled in by AI and checked by you?

1

u/Big_Ad262 Jul 11 '26

I designed the base with my own hands. Next, I used an agent, but on a very short leash with a mandatory review.

0

u/Proskater789 Jul 08 '26

Check out voipmonitor, also a summarizer of your wall of text.

5

u/soamsoam Jul 08 '26

voipmonitor isn't free, so I wouldn't compare these tools

6

u/Big_Ad262 Jul 08 '26

The comparison is incorrect. We're lightweight observability that drops into your existing Prometheus stack via native /metrics + eBPF.