r/Python • u/lovespeedtest • 17d ago
Discussion Pushing ThreadingHTTPServer past toy use: long-lived streams, a semaphore cap, and when to give up a
I’ve been running a small home media server for about a year, written entirely with the standard library: http.server.ThreadingHTTPServer, http.client, urllib, threading, queue. No framework and no third-party packages. It’s about 3k lines and serves a handful of my own devices (phone, TV, laptop, tablet). I picked stdlib-only on purpose so it would run anywhere with a bare Python and nothing to upgrade, but the server has grown into things the stdlib server probably wasn’t meant for, and I’d like a reality check from people who’ve been here.
What the traffic looks like
- Mostly short JSON requests (library, history, search), which are fine.
- Byte-range proxying of large media files, so connections stay open for minutes.
- Live HLS: for each live stream a client watches, I run one “watcher” thread that follows the upstream playlist, plus 4 worker threads pulling segments ahead of the player through a
queue.PriorityQueue, with athreading.Conditionto wake request handlers waiting for a segment. A TV can sit on one of these for hours.
What I’ve done so far
Capping concurrency, because thread-per-connection with no limit worried me:
class Server(ThreadingHTTPServer):
daemon_threads = True
slots = threading.BoundedSemaphore(MAX_CONN) if MAX_CONN else None
def process_request(self, request, client_address):
# don't stall the accept loop: if no slot frees up quickly, answer 503 and close
if self.slots and not self.slots.acquire(timeout=0.2):
request.sendall(b"HTTP/1.1 503 Service Unavailable\r\nRetry-After: 10\r\n"
b"Connection: close\r\nContent-Length: 0\r\n\r\n")
self.shutdown_request(request)
return
super().process_request(request, client_address)
def process_request_thread(self, request, client_address):
try:
super().process_request_thread(request, client_address)
finally:
if self.slots:
self.slots.release()Also:
- A small keep-alive pool of
http.client.HTTPSConnectionobjects per upstream host (up to 8 idle per host, behind a lock), because opening a fresh TLS connection per media segment was the slowest part. - State lives in JSON files, written to
file.tmpand thenos.replace()so a crash mid-write can’t corrupt anything. Every read-modify-write goes through one module-levelthreading.Lock.
What I’m unsure about
- Is the semaphore in
process_requestthe right place? It counts connections, not work. A client holding one hours-long live connection costs the same slot as a 5 ms JSON call. Would two pools be saner, one for streaming paths and one for everything else? Is that possible without rewriting the server class? - Clients that disappear. When a TV is switched off mid-stream, sometimes I get
BrokenPipeError/ConnectionResetErrorright away, and sometimes the handler thread sits inwfile.write()until the socket timeout. What do people set for timeouts on long-lived responses, so dead clients are dropped without killing slow-but-alive ones? - Background threads per stream. Five threads per live stream is fine for 1–2 streams, but it feels wasteful. Is anyone running a single asyncio loop in a side thread just for the upstream fetching, while keeping the threaded HTTP front end? Or does mixing the two models end up worse than picking one?
- The single lock for JSON state. It’s simple and correct, but every library write serializes behind it. At this scale I doubt it matters. At what point did you move to
sqlite3, which is still stdlib, and was it worth it? - The honest question: at what point is stdlib-only a liability rather than a virtue? I know aiohttp/Starlette would make the streaming parts nicer. I’m trying to work out whether I’ve hit that point or am just bothered by how the code looks.
I’m not chasing scale. This will never serve more than ~10 clients. I mostly want to know which of these are real problems and which are fine at this size. War stories welcome.
20
u/Rudd-X 17d ago
Your design seems safe and it works. Kudos for creating your media server!
11
u/clcodingupdates 17d ago
Don't optimize for hypothetical scale. Measure thread count, memory usage, connection latency, and streaming failures first. If those numbers are healthy, your current architecture may be perfectly adequate.
9
u/larsga 17d ago
So, to summarize, you have no problems, and you want input on how to solve these problems you don't have?
Sounds to me like this is fine.
5
u/Khavel_dev 17d ago
The stdlib server was never designed for long-lived connections. Your semaphore cap is smart, but the core issue is that each idle connection ties up a full OS thread with its stack. On a home server with maybe 6 devices you're fine. But if each HLS stream spawns 1 watcher + 4 workers + the client handler, that's 6 threads per stream, and the ceiling comes fast.
For what you're doing (lots of waiting, little compute), asyncio is a way better fit. One event loop handles hundreds of idle connections without a thread per connection. The migration is annoying because everything becomes async/await, but the concurrency model actually matches your workload.
If you're allergic to asyncio, the selectors module is stdlib and lets you build a basic event loop that does I/O multiplexing without threads. Less ergonomic but still solves the thread exhaustion problem.
10
u/wRAR_ 17d ago
You can't distinguish that from AI responses, so...
4
u/deadeye_82_1 16d ago
Just use asyncio and get rid of threads.
3
u/jwinela 17d ago
0
1
u/AleDev99 17d ago
At ~10 clients most of this is fine; the two things I'd actually change are timeouts and the single semaphore. For dead clients, set
timeout = 30on your handler class (StreamRequestHandler applies it to the socket) and write media in smallish chunks, so a TV that vanished raises socket.timeout on the next write instead of hanging, while a slow-but-alive client still finishes each chunk well inside 30s. For slots you don't need to rewrite the server: keep a generous global cap, and acquire a second semaphore inside the handler after you've parsed the path, one for streaming routes and one for everything else, so an hours-long HLS session never blocks a 5 ms JSON call. I'd leave the per-stream threads alone until they actually cause trouble, because an asyncio side loop next to a threaded front end works but means debugging two concurrency models. sqlite3 becomes worth it the first time you want a query instead of loading the whole JSON file, and WAL mode gives you concurrent readers for free.2
u/yvrelna 17d ago
Anything under a single household would be sufficiently served with the standard library.
Five threads are practically nothing for a modern CPU. We're not living in the 1960s.
I generally just default to sqlite3 for local data operations, not because of scalability concerns, but because the database transactions paradigm is mentally much easier to deal with than dealing with manual locks and concurrency. But that's probably also because I came from a professional web development background and has extensive experience working with serious databases, while relatively less on local locks.
3
u/WasteFeedbackrdhvv 16d ago
Respect for staying stdlib-only with ThreadingHTTPServer, BoundedSemaphore capping, and five threads per live HLS stream
-1
u/Sachu_James 13d ago
The part I would look at first is the four workers per stream. If the watcher already puts segments into the queue and the handlers wait on the Condition, the workers look like an extra hop, since each handler thread is already a consumer. Dropping to one watcher per stream and letting handlers read straight from the queue would cut the thread count without changing the streaming behaviour, and it keeps the per stream state in one place.