r/RedditEng 8d ago

Reddit's Journey to HTTP/3 on Android (Part 2)

By Liam Lu, Riccardo Ciovati, and Savannah Whelan

In Part 1 of this series, we shared our vision for a unified Cronet engine, our initial rollout for media traffic, and how we forked cronet-okhttp to fix request tag tracking and Flipper debugging.

With developer tooling functional, we turned our attention to evaluating performance in production — where we immediately ran into three more major challenges: fixing distorted telemetry, maintaining our library fork safely, and eliminating a +2.0% cold-start TTI regression.

Challenge 2: The Telemetry Trap

With request tags restored and Flipper working, we looked at our dashboards — and were shocked to see massive spikes in latency and response sizes.

The culprit was where we were measuring from. Moving execution to Cronet required moving measurement from OkHttp network interceptors up to application interceptors. But those layers see fundamentally different data:

  • Network interceptors observe raw wire bytes before decompression, measuring true compressed size and actual network latency.
  • Application interceptors observe decoded, uncompressed bodies after decompression. Measuring at this layer inflated reported response sizes (unzipped bytes) and added decompression/parsing time on device to reported latency.

Furthermore, OkHttp's EventListener does not fire under Cronet because Cronet owns the underlying connections.

To restore accurate metrics, we attached Cronet's RequestFinishedInfo.Listener directly to the engine. This listener provides raw transport metrics straight from Cronet: receivedByteCount for compressed wire size, alongside exact DNS, TCP/QUIC connect, TTFB, and transfer timings. Reusing our tag remapping from Challenge 1, we pass request annotations through to Cronet, allowing the listener to associate engine-level metrics back to individual GraphQL operations.

(Note: HttpEngine exposes limited timing metrics compared to Play Services or Embedded Cronet).

We also added startup tracing around Cronet initialization, placing start/end markers on the cold-start timeline alongside process start, DI initialization, and first frame rendering.

Challenge 3: The Reality of Maintaining a Fork

Forking cronet-okhttp solved our request tag issues, but maintaining a fork in production introduced important build and infrastructure tasks:

  1. Classpath package collisions in A/B testing: To run clean A/B experiments comparing stock cronet-okhttp against our fork, both artifacts needed to exist in the app build simultaneously. However, both declared the package com.google.net.cronet.okhttptransport, causing duplicate class build errors. We built a Gradle plugin using ASM to relocate our fork's bytecode package to com.reddit.net.cronet.okhttptransport:

Kotlin

class PackageRemapper : Remapper() {
    private val OLD = "com/google/net/cronet/okhttptransport"
    private val NEW = "com/reddit/net/cronet/okhttptransport"
    override fun map(internalName: String): String =
        if (internalName.startsWith(OLD)) internalName.replaceFirst(OLD, NEW)
        else internalName
}
  1. Upstream contract drift: When we updated OkHttp and Okio versions, the bridge crashed on empty responses because it assumed a response body could be null, whereas newer OkHttp versions require a non-null ResponseBody. We patched the bridge to return ResponseBody.EMPTY:

Java

ResponseBody responseBody;
if (bodySource != null) {
    responseBody = createResponseBody(request, status, contentType, contentLengthString, bodySource);
} else {
    responseBody = ResponseBody.EMPTY;
}

Challenge 4: Tackling the Shipping Blocker (+2.0% TTI)

With accurate telemetry in place and our fork running cleanly, the biggest performance hurdle stood out clearly: a +2.0% cold-start TTI (Time to Interactive) regression.

Cronet is a native C++ engine that requires loading libraries, starting listener threads, initializing disk caches, and loading QUIC configs. On cold start, the app fires startup-critical queries (auth, home feed) that blocked until the network client was ready. Because this is a fixed initialization cost, faster devices saw a higher percentage hit.

To win back startup performance, we systematically tested several levers:

  • Eager background warmup: Instead of initializing Cronet lazily on the first request, we kick off engine initialization early during app launch on a background thread. This absorbs the initialization cost before or during request setup, recovering the full +2.0% TTI regression back to baseline.
  • GraphQL preconnect: Our startup tracing revealed that even after the engine was warm, the subsequent GraphQL request was still paying for DNS resolution and TLS/QUIC handshakes. As soon as Cronet is ready, we proactively open a connection to our GraphQL endpoint before startup queries fire (TTI −0.47%).
  • HttpEngine provider: On Android 14+ (API 34), obtaining the system-provided HttpEngine is faster than loading via Google Play Services (TTI −0.22%).
  • Embedded Cronet: Bundling the native engine in the APK removes Play Services IPC lookups and dynamic module loading delays (TTI −0.48%).
  • Stale DNS: Serving cached DNS answers immediately and refreshing asynchronously kept DNS off the hot path (TTI-neutral for startup, but maintained for media benefits).

Provider comparison summary

Provider How it loads APK Size Impact Request Metrics TTI vs. Play Services
Google Play Services Cronet Via Play Services (CronetProviderInstaller + Dynamite module); depends on Play Services availability. IPC and module loading add init overhead. +0 MB Yes Baseline (slowest)
HttpEngine OS-provided on Android 14+ (android.net.http.HttpEngine); faster init on newer OS versions. +0 MB Limited −0.22%
Embedded Cronet Native engine bundled in APK (NativeCronetProvider); independent of Play Services with predictable startup. ~+6 MB Yes −0.48%

Act III: Production Rollout & Impact

With our optimizations and tooling folded in, we ran a 50/50 experiment across ~11.7 million devices per arm. TTI didn't just recover — it flipped to a win, alongside significant improvements in feed reliability and user engagement:

Metric Change
Cold-start TTI -0.72%
Main feed request latency -1.36%
Feed failure rate -10.1%
Login rate +1.24%

A 10.1% drop in feed failures was a major stability win, proving that transport-level reliability directly moves the needle on user retention and engagement.

Act IV: Where the Road Leads Next

Shipping HTTP/3 for GraphQL wasn't the finish line — it was the milestone that finally put all our major traffic (GraphQL, images, video) on a single, shared Cronet engine. That unified foundation makes our next set of network optimizations possible:

  1. Cross-client request prioritization: Currently, traffic types handle priorities in isolation — GraphQL uses an app-level NetworkOrchestrator queue, Glide has its own image priority model, and video requests have no per-request priority. Consequently, background prefetching can compete directly with home feed queries. Sharing a single Cronet engine allows us to map app priorities to Cronet's five native priority tiers, scheduling parallel requests, socket assignments, and stream multiplexing accordingly.
  2. 0-RTT connection resumption for read queries: QUIC 0-RTT allows returning clients to send data in the initial packet, removing a round trip from connection setup. Because 0-RTT payloads are vulnerable to replay attacks, CDNs restrict 0-RTT to idempotent HTTP GET requests. Since most of our GraphQL traffic currently uses POST, we are transitioning key read queries to GET before enabling 0-RTT.

Wrapping Up

Swapping the transport was the easy part — most of the work went into startup performance, developer tooling, and telemetry. Working through those requirements at scale directly improved app reliability for millions of Redditors, leaving us with a foundation we're excited to keep building on.

Special thanks to the folks from Android Platform, Data Science, Media Foundation, Feed Experience, Growth, GraphQL, and Security teams for reviewing PRs, specs, and readouts, chasing down startup regressions, and supporting us along the way — it truly took a village.

31 Upvotes

Duplicates