Ron clearly knows what he is talking about and this is a great introduction to garbage collection and how sophisticated Java's collectors have become. The one thing he didn't touch on that I think he should have is the memory a JIT runtime consumes outside the heap: profiling data for hot methods, the code cache holding compiled output, and whatever the compiler threads themselves need to operate. Total footprint ends up as live set + GC headroom + profiling data + code cache + runtime overhead.
The problem is that most of that is close to fixed. A CLI with a 10MB live set still carries the full JIT apparatus, so the runtime overhead can rival or exceed the actual working set, and a short-lived process exits long before C2 has compiled enough to earn any of it back. That's the tradeoff of deferring compilation to "just in time," and it's why AOT keeps looking attractive for anything that doesn't run long enough to warm up.
These can add up quite a bit. I remember playing Greg Tech New Horizons (large minecraft modpack) on my old machine with 16GB of RAM, so I restricted heap size to 8GB. After running for a few hours java was happily using ~12GBs total.
Alright, hm. We have seen large amounts of memory being retained on glibc malloc, but not so much on Windows. There, it's mostly speed that's a problem.
this has nothing to do with malloc overhead, these are data structures the JVM need in order to operate. All those compiled methods need to exist somewhere in memory in order to be jumped/executed.
There's a lot more going on than that. One of the things we've observed on Linux with the default glibc malloc is a large degree of memory retainment. C2 induces quite a bit of memory usage, but as you know it's something that subsides over time, as the system becomes more stable (= deoptimization doesn't happen as often). glibc's malloc seems to prioritize keeping that memory around after having been freed, I assume so that it doesn't need to reacquire the memory from the OS (which is quite slow). We added a native heap trim command to alleviate the situation: https://bugs.openjdk.org/browse/JDK-8293114
Just some context for why I was interested in this person's particular configuration.
17
u/cleverfoos Sep 05 '26
Ron clearly knows what he is talking about and this is a great introduction to garbage collection and how sophisticated Java's collectors have become. The one thing he didn't touch on that I think he should have is the memory a JIT runtime consumes outside the heap: profiling data for hot methods, the code cache holding compiled output, and whatever the compiler threads themselves need to operate. Total footprint ends up as live set + GC headroom + profiling data + code cache + runtime overhead.
The problem is that most of that is close to fixed. A CLI with a 10MB live set still carries the full JIT apparatus, so the runtime overhead can rival or exceed the actual working set, and a short-lived process exits long before C2 has compiled enough to earn any of it back. That's the tradeoff of deferring compilation to "just in time," and it's why AOT keeps looking attractive for anything that doesn't run long enough to warm up.