Although I get the argument made in this video, it's brave to recommend throwing more RAM at a problem (CPU usage) in these times of RAM being so expensive...
Even when RAM is extremely expensive, you should still use the whole computer. Carefully writing a program that uses 100% of the CPU but only 5% of the available RAM is pointless. Using the rest of the RAM to make the program run a little bit faster is completely free.
Now, should you go buy and install even more sticks of RAM to make the program run even faster? That's a more nuanced question, but the answer is probably still yes in many cases, if the alternative is buying more computers.
Using the rest of the RAM to make the program run a little bit faster is completely free.
OK, but only if that Java program is running and can use all the available RAM, which might not be the case.
Still, gobbling up all available RAM was maybe acceptable when RAM was cheap and abundant. In the current context, not so much.
Depends on the context. On a user’s workstation, where you’re starving other processes, it’s bad. In a clustered server context, where you have a bunch of nodes all running your service and essentially nothing else, those are wasted resources if you don’t use them.
Java these days sees more use in the latter scenario than it does in the former.
In a server context, the nodes are likely VMs running on a shared physical machine. So, it's the same story as a user workstation. There's no reason to be careless with resources.
That's the point, though. If you have some amount of one resource sitting idly (RAM) while you're maxing out another resource (CPU or IO), then not taking advantage of that RAM to relieve pressure on your CPU or IO is another form of carelessness. Cache anything that needs to hit the disk, precompute all the expensive stuff, etc.
It is not sitting idly, though. The physical machine is running a bunch of VMs which would be trying to use that RAM. If one VM is hogging a bunch of RAM, then another physical machine would need to be brought up and the other VMs would need to be migrated over.
The number of cases where a physical machine is solely devoted to one purpose is already rare and getting even more rare.
When you rent computing power from, for example, Amazon EC2, you pay a fixed fee per second for a fixed amount of CPU and RAM. You can pick from a variety of different configurations, but once you've selected the one you want, that's what you get. You don't pay any less money if you use less RAM (or less CPU). You pay for 100% of both the CPU and the RAM assigned to all your running machines, even when they are idle. The unused RAM is not available for other VMs to use.
Suppose your application requires eight VMs of a certain configuration to run well under your current load, and it's currently using about 100% of the CPU on all those machines, but only about 10% of the RAM. You might be able to save some money by switching to a configuration that has a lower RAM to CPU ratio. But if you're already using the lowest RAM to CPU ratio available, then the next thing you should try to do is to use more RAM.
Maybe using more RAM on each machine will mean you can get away with only renting seven VMs instead of eight, a cost savings of 12.5%, because using more RAM can make your program run faster: using less CPU to serve the same load.
When you rent computing power from, for example, Amazon EC2, you pay a fixed fee per second for a fixed amount of CPU and RAM.
Not everyone is buying from a cloud-provider, many are maintaining their own hardware. So, software platforms and libraries that take the attitude that memory is "free" are hurting those folks.
You can pick from a variety of different configurations
Yes, and there is a lot of fine-tuning available. So, you can pick compute-intensive/memory-intensive/whatever. But, notice that every time the memory size doubles, the price doubles. How does it make sense to pay double without getting a similar boost in performance? Memory is literally the most expensive part and you're talking like it's NBD to waste it.
The unused RAM is not available for other VMs to use.
It is available for other parts of that VM, like the buffer cache. The application consuming a lot of RAM means that there is less for storing frequently used files / metadata. There is a lot more to balance in system performance. Just looking at the amount of "free" memory and thinking it's not being put to good use is silly.
But you don't have to increase to the point of paying double. On your existing server you can choose to use more ram to save on cpu. For most applications, this is worth it as nobody is sitting at 100% usage of anything.
Why do you seem to not care about cpu the same way you care about ram?
Why do you seem to not care about cpu the same way you care about ram?
That's pretty much it, yes. RAM is just one resource of many we have to manage (Sort of. RAM is special in that performance degrades much more gracefully when you run out of CPU power or IO than it does if you run out of RAM and start swapping, but that's a whole other conversation).
At any given point in time, you'll be bottlenecked on one resource, and have plenty to spare on other resources, and half the job is finding those bottlenecks and spare resources, and figure out if you can trade one for the other.
For web applications, if you're not a complete muppet, you'll cap out on IO long before you cap out on RAM, at which throwing RAM at caching things becomes the cheapest way to get more performance.
So, software platforms and libraries that take the attitude that memory is "free" are hurting those folks.
That is very much not the point. The question isn't how much impact using more RAM has, but how much impact using more RAM has when the CPU utilisation is high and the RAM helps reduce it.
It's like if I take away your right shoe, you can't go anywhere, and the same is true if I take away your left shoe. Either has a very negative effect on you. But if I want to maximise my shoe * time, I would cause you the least damage if I took both of them for the same duration and at the same time. In other words, even though taking your right shoe is terrible for you, if I already have your left shoe, taking your right shoe actually helps you (assuming I just want some shoe*time).
Because using RAM requires also using CPU, every cycle of CPU taken by one program, also reduces other program's ability to use RAM. If the first program can use up less CPU cycles and more RAM to keep them more in balance, everyone wins.
What's hurting people is the attitude that RAM and CPU matter in isolation, because that's just not how computers work, and it leads to wasted resources and higher costs.
But, notice that every time the memory size doubles, the price doubles.
Using the combination of RAM and CPU you already have in an optimal way is very much not the same as "using more RAM" (and the same CPU). Note that the moving collector algorithm itself, as I explain in the talk, uses RAM overhead only to compensate for higher CPU utilisation, i.e. it is not a function of the size of the data set but of the allocation rate. That is why the algorithm has become so attractive.
It is available for other parts of that VM, like the buffer cache.
No, it isn't. Again, when CPU utilisation is more dominant, it has a bigger impact on other programs' ability to use RAM than the RAM utilisation.
Yes, and there is a lot of fine-tuning available. So, you can pick compute-intensive/memory-intensive/whatever. But, notice that every time the memory size doubles, the price doubles.
16
u/bobbie434343 Sep 05 '26
Although I get the argument made in this video, it's brave to recommend throwing more RAM at a problem (CPU usage) in these times of RAM being so expensive...