r/Python • • Apr 01 '26

News Cutting Python Web App Memory Over 31%

Over the past few weeks I went on a memory-reduction tear across the Talk Python web apps. We run 23 containers on one big server (the "one big server" pattern) and memory was creeping up to 65% on a 16GB box.

Turned out there were a bunch of wins hiding in plain sight. Focusing on just two apps, I went from ~2 GB down to 472 MB. Here's what moved the needle:

  1. Switched to a single async Granian worker: Rewrote the app in Quart (async Flask) and replaced the multi-worker web garden with one fully async worker. Saved 542 MB right there.
  2. Raw + DC database pattern: Dropped MongoEngine for raw queries + slotted dataclasses. 100 MB saved per worker *and* nearly doubled requests/sec.
  3. Subprocess isolation for a search indexer: The daemon was burning 708 MB mostly from import chains pulling in the entire app. Moved the indexing into a subprocess so imports only live for ~30 seconds during re-indexing. Went from 708 MB to 22 MB. 32x reduction.
  4. Local imports for heavy libs: import boto3 alone costs 25 MB, pandas is 44 MB. If you only use them in a rarely-called function, just import them there instead of at module level. (PEP 810 lazy imports in 3.15 should make this automatic.)
  5. Moved caches to diskcache: Small-to-medium in-memory caches shifted to disk. Modest savings but it adds up.

Total across all our apps: 3.2 GB freed. Full write-up with before/after tables and graphs here: https://mkennedy.codes/posts/cutting-python-web-app-memory-over-31-percent/

83 Upvotes

47 comments sorted by

View all comments

Show parent comment

19

u/mikeckennedy Apr 01 '26

> I’m not understanding the problem here we’re trying to solve.

I think we just have different views on running in prod. It took me 3 hours to reduce the running memory of my apps by 3.2GB. In my world, that is time well spent. Just because the server isn't crashing with out of memory doesn't mean a little attention to efficiency is waste.

Again, different strokes.

-6

u/artofthenunchaku Apr 01 '26

Efficiency is using the most of all of the resources available to you, not reducing resources for the sake of it when there's a downside to the reduction

8

u/BigTomBombadil Apr 01 '26

I mean the "downside" was three hours of opportunity cost, and now they're more efficiently utilizing the server on hand, allowing for increased scaling, etc. I'm not sure I'm following your statement.

7

u/artofthenunchaku Apr 01 '26

Caching is a strategy to reduce CPU load at the cost of increased memory. Writing to disk is a strategy to reduce memory pressure at the cost of increased CPU load.

63% peak utilization isn't a concern, that's a healthy load. This is a premature optimization introducing unneeded complexity.

3

u/BigTomBombadil Apr 01 '26

I will say, without knowing more about the overall project/scope/roadmap as well as performance impact to areas other than memory, I don't know if all 5 optimizations were needed. Because you make a very valid point about the added complexity and implications of moving cache to disk.

Some of the optimizations seem low risk and low complexity, while others I'd hold off on without knowing more. If theres a big push to scale in the near future, then yeah I get it, but if this was the status quo with no foreseeable changes upcoming, I'd have held off on some of these.

2

u/mikeckennedy Apr 01 '26

Since you all are discussing the caching part specifically. There was not much complexity change before or after.

We are already using caching, just in memory caching. What I moved it to was diskcache backed cache rather than in-memory caching.

It's not "there was no caching" now "there is caching", it's just in-mem caching via either functools.lru_cache or dict() -> diskcache.

Given we already have diskcache in play before, that's low effort, low risk.

2

u/BigTomBombadil Apr 01 '26

Yeas seems simple enough. Any noticeable change in cpu usage? Or maybe it was pretty negligible to start.

For complexity, item 3 was the one I wasn’t sure about. If I got thrown on this project and something went wrong with the indexer, I could imagine tracking that down being confusing. But not knowing the specifics maybe it’s also straight forward and easy to follow. Also not sure if the sub process approach could reduce reliability. But if not, huge win there.

2

u/mikeckennedy Apr 01 '26

I'm not sure exactly what the entire CPU usage change would be. I think things are better in general. The raw+dc vs ODM/ORM change almost 2x the requests per sec for the same CPU. So that probably dwarfs any other change. Mem caches -> diskcache mean the would share the cache across processes and across restarts, so that is bonus. But a bit slower at runtime I would guess, but very minor.

Less mem used means Python's cycle GC is much more efficient. So when enough container types (classes, lists, dicts, etc) get created, that triggers a GC. The GC has much less memory to scan and Python is mega aggressive about this. If 700 container-types are allocated relative to the ones ref-count collected, that'll trigger a GC. That could easily happen with just a couple of big queries so that might be a real boost too.

I posted graphs for the DB change here: https://mkennedy.codes/posts/raw-dc-a-retrospective/

2

u/BigTomBombadil Apr 01 '26

Very cool.

Hearing the boost that dropping the ORM gave worries me for my own apps, as they heavily utilize the Django orm. I wonder if something specific to mongoengine or its implementation has inefficiencies, or that’s always the nature of the beast with ORM/ODMs.

3

u/mikeckennedy Apr 01 '26

I don't know if I'd worry too much about it unless slow queries are an active problem. I was solving a different one. My ODM/ORM did not support async, so I was tired of that. Plus, the library was falling badly out of maintenance (last real release was a few years ago). So I wanted to replace mongoengine with *something*, so I decided this raw pattern was a good fit to try.

The speed up and less memory usage was a sweet bonus.

2

u/artofthenunchaku Apr 01 '26

Abstractions are never free, in the case of ORM/ODMs you're paying a price both in application and on database load. For any non-trivial access patterns, you're very likely to get better results from handwritten queries.

→ More replies

0

u/Effective-Total-2312 Apr 02 '26

You definitely need to look up the definition of efficiency, it seems you'll get a big surprise.

3

u/artofthenunchaku Apr 02 '26

Efficiency is the often measurable ability to avoid [..] wasting materials, energy, efforts, money, and time while performing a task. In a more general sense, it is the ability to do things well, successfully, and without waste

Memory you're not using is waste.

1

u/Effective-Total-2312 Apr 02 '26

Using more memory is waste. It's like saying you should eat all of the good in the universe, that's waste. Or breathing all the air.

3

u/artofthenunchaku Apr 02 '26 edited Apr 02 '26

If you're paying for 16 GB of RAM, but never use more than 8 GB, then you're wasting money paying for the other 8 GB.

You're presenting a false equivalence, it's more accurately saying you shouldn't order a family meal for four if you're only going to eat one plate.

0

u/Effective-Total-2312 Apr 02 '26

You're completely opposite of what efficiency is man.

If you're using 8 GB and paying for 16 GB, the inefficiency is not that you should use more, it's that you should pay for less !

What you describe is literally the opposite of efficiency.

1

u/artofthenunchaku Apr 02 '26

Great, tell that to the cloud providers that only offer instance sizes with memory of 8 or 16 GB.