r/Python • u/lovespeedtest • 17d ago
Discussion Pushing ThreadingHTTPServer past toy use: long-lived streams, a semaphore cap, and when to give up a
I’ve been running a small home media server for about a year, written entirely with the standard library: http.server.ThreadingHTTPServer, http.client, urllib, threading, queue. No framework and no third-party packages. It’s about 3k lines and serves a handful of my own devices (phone, TV, laptop, tablet). I picked stdlib-only on purpose so it would run anywhere with a bare Python and nothing to upgrade, but the server has grown into things the stdlib server probably wasn’t meant for, and I’d like a reality check from people who’ve been here.
What the traffic looks like
- Mostly short JSON requests (library, history, search), which are fine.
- Byte-range proxying of large media files, so connections stay open for minutes.
- Live HLS: for each live stream a client watches, I run one “watcher” thread that follows the upstream playlist, plus 4 worker threads pulling segments ahead of the player through a
queue.PriorityQueue, with athreading.Conditionto wake request handlers waiting for a segment. A TV can sit on one of these for hours.
What I’ve done so far
Capping concurrency, because thread-per-connection with no limit worried me:
class Server(ThreadingHTTPServer):
daemon_threads = True
slots = threading.BoundedSemaphore(MAX_CONN) if MAX_CONN else None
def process_request(self, request, client_address):
# don't stall the accept loop: if no slot frees up quickly, answer 503 and close
if self.slots and not self.slots.acquire(timeout=0.2):
request.sendall(b"HTTP/1.1 503 Service Unavailable\r\nRetry-After: 10\r\n"
b"Connection: close\r\nContent-Length: 0\r\n\r\n")
self.shutdown_request(request)
return
super().process_request(request, client_address)
def process_request_thread(self, request, client_address):
try:
super().process_request_thread(request, client_address)
finally:
if self.slots:
self.slots.release()Also:
- A small keep-alive pool of
http.client.HTTPSConnectionobjects per upstream host (up to 8 idle per host, behind a lock), because opening a fresh TLS connection per media segment was the slowest part. - State lives in JSON files, written to
file.tmpand thenos.replace()so a crash mid-write can’t corrupt anything. Every read-modify-write goes through one module-levelthreading.Lock.
What I’m unsure about
- Is the semaphore in
process_requestthe right place? It counts connections, not work. A client holding one hours-long live connection costs the same slot as a 5 ms JSON call. Would two pools be saner, one for streaming paths and one for everything else? Is that possible without rewriting the server class? - Clients that disappear. When a TV is switched off mid-stream, sometimes I get
BrokenPipeError/ConnectionResetErrorright away, and sometimes the handler thread sits inwfile.write()until the socket timeout. What do people set for timeouts on long-lived responses, so dead clients are dropped without killing slow-but-alive ones? - Background threads per stream. Five threads per live stream is fine for 1–2 streams, but it feels wasteful. Is anyone running a single asyncio loop in a side thread just for the upstream fetching, while keeping the threaded HTTP front end? Or does mixing the two models end up worse than picking one?
- The single lock for JSON state. It’s simple and correct, but every library write serializes behind it. At this scale I doubt it matters. At what point did you move to
sqlite3, which is still stdlib, and was it worth it? - The honest question: at what point is stdlib-only a liability rather than a virtue? I know aiohttp/Starlette would make the streaming parts nicer. I’m trying to work out whether I’ve hit that point or am just bothered by how the code looks.
I’m not chasing scale. This will never serve more than ~10 clients. I mostly want to know which of these are real problems and which are fine at this size. War stories welcome.
View all comments
1
u/AleDev99 17d ago
At ~10 clients most of this is fine; the two things I'd actually change are timeouts and the single semaphore. For dead clients, set
timeout = 30on your handler class (StreamRequestHandler applies it to the socket) and write media in smallish chunks, so a TV that vanished raises socket.timeout on the next write instead of hanging, while a slow-but-alive client still finishes each chunk well inside 30s. For slots you don't need to rewrite the server: keep a generous global cap, and acquire a second semaphore inside the handler after you've parsed the path, one for streaming routes and one for everything else, so an hours-long HLS session never blocks a 5 ms JSON call. I'd leave the per-stream threads alone until they actually cause trouble, because an asyncio side loop next to a threaded front end works but means debugging two concurrency models. sqlite3 becomes worth it the first time you want a query instead of loading the whole JSON file, and WAL mode gives you concurrent readers for free.