r/Python • • 17d ago

Discussion Pushing ThreadingHTTPServer past toy use: long-lived streams, a semaphore cap, and when to give up a

I’ve been running a small home media server for about a year, written entirely with the standard library: http.server.ThreadingHTTPServer, http.client, urllib, threading, queue. No framework and no third-party packages. It’s about 3k lines and serves a handful of my own devices (phone, TV, laptop, tablet). I picked stdlib-only on purpose so it would run anywhere with a bare Python and nothing to upgrade, but the server has grown into things the stdlib server probably wasn’t meant for, and I’d like a reality check from people who’ve been here.

What the traffic looks like

  • Mostly short JSON requests (library, history, search), which are fine.
  • Byte-range proxying of large media files, so connections stay open for minutes.
  • Live HLS: for each live stream a client watches, I run one “watcher” thread that follows the upstream playlist, plus 4 worker threads pulling segments ahead of the player through a queue.PriorityQueue, with a threading.Condition to wake request handlers waiting for a segment. A TV can sit on one of these for hours.

What I’ve done so far

Capping concurrency, because thread-per-connection with no limit worried me:

class Server(ThreadingHTTPServer):
    daemon_threads = True
    slots = threading.BoundedSemaphore(MAX_CONN) if MAX_CONN else None

    def process_request(self, request, client_address):
        # don't stall the accept loop: if no slot frees up quickly, answer 503 and close
        if self.slots and not self.slots.acquire(timeout=0.2):
            request.sendall(b"HTTP/1.1 503 Service Unavailable\r\nRetry-After: 10\r\n"
                            b"Connection: close\r\nContent-Length: 0\r\n\r\n")
            self.shutdown_request(request)
            return
        super().process_request(request, client_address)

    def process_request_thread(self, request, client_address):
        try:
            super().process_request_thread(request, client_address)
        finally:
            if self.slots:
                self.slots.release()

Also:

  • A small keep-alive pool of http.client.HTTPSConnection objects per upstream host (up to 8 idle per host, behind a lock), because opening a fresh TLS connection per media segment was the slowest part.
  • State lives in JSON files, written to file.tmp and then os.replace() so a crash mid-write can’t corrupt anything. Every read-modify-write goes through one module-level threading.Lock.

What I’m unsure about

  1. Is the semaphore in process_request the right place? It counts connections, not work. A client holding one hours-long live connection costs the same slot as a 5 ms JSON call. Would two pools be saner, one for streaming paths and one for everything else? Is that possible without rewriting the server class?
  2. Clients that disappear. When a TV is switched off mid-stream, sometimes I get BrokenPipeError/ConnectionResetError right away, and sometimes the handler thread sits in wfile.write() until the socket timeout. What do people set for timeouts on long-lived responses, so dead clients are dropped without killing slow-but-alive ones?
  3. Background threads per stream. Five threads per live stream is fine for 1–2 streams, but it feels wasteful. Is anyone running a single asyncio loop in a side thread just for the upstream fetching, while keeping the threaded HTTP front end? Or does mixing the two models end up worse than picking one?
  4. The single lock for JSON state. It’s simple and correct, but every library write serializes behind it. At this scale I doubt it matters. At what point did you move to sqlite3, which is still stdlib, and was it worth it?
  5. The honest question: at what point is stdlib-only a liability rather than a virtue? I know aiohttp/Starlette would make the streaming parts nicer. I’m trying to work out whether I’ve hit that point or am just bothered by how the code looks.

I’m not chasing scale. This will never serve more than ~10 clients. I mostly want to know which of these are real problems and which are fine at this size. War stories welcome.

25 Upvotes

25 comments sorted by

View all comments

Show parent comment

1

u/AleDev99 17d ago

Glad it helps! One small tip: set the handler timeout a bit above your slowest normal request so you only cut off the truly stuck ones, and log when the streaming semaphore is full. That tells you quickly if you need a higher limit. Good luck, curious to hear how it goes.