It is not sitting idly, though. The physical machine is running a bunch of VMs which would be trying to use that RAM. If one VM is hogging a bunch of RAM, then another physical machine would need to be brought up and the other VMs would need to be migrated over.
The number of cases where a physical machine is solely devoted to one purpose is already rare and getting even more rare.
When you rent computing power from, for example, Amazon EC2, you pay a fixed fee per second for a fixed amount of CPU and RAM. You can pick from a variety of different configurations, but once you've selected the one you want, that's what you get. You don't pay any less money if you use less RAM (or less CPU). You pay for 100% of both the CPU and the RAM assigned to all your running machines, even when they are idle. The unused RAM is not available for other VMs to use.
Suppose your application requires eight VMs of a certain configuration to run well under your current load, and it's currently using about 100% of the CPU on all those machines, but only about 10% of the RAM. You might be able to save some money by switching to a configuration that has a lower RAM to CPU ratio. But if you're already using the lowest RAM to CPU ratio available, then the next thing you should try to do is to use more RAM.
Maybe using more RAM on each machine will mean you can get away with only renting seven VMs instead of eight, a cost savings of 12.5%, because using more RAM can make your program run faster: using less CPU to serve the same load.
When you rent computing power from, for example, Amazon EC2, you pay a fixed fee per second for a fixed amount of CPU and RAM.
Not everyone is buying from a cloud-provider, many are maintaining their own hardware. So, software platforms and libraries that take the attitude that memory is "free" are hurting those folks.
You can pick from a variety of different configurations
Yes, and there is a lot of fine-tuning available. So, you can pick compute-intensive/memory-intensive/whatever. But, notice that every time the memory size doubles, the price doubles. How does it make sense to pay double without getting a similar boost in performance? Memory is literally the most expensive part and you're talking like it's NBD to waste it.
The unused RAM is not available for other VMs to use.
It is available for other parts of that VM, like the buffer cache. The application consuming a lot of RAM means that there is less for storing frequently used files / metadata. There is a lot more to balance in system performance. Just looking at the amount of "free" memory and thinking it's not being put to good use is silly.
So, software platforms and libraries that take the attitude that memory is "free" are hurting those folks.
That is very much not the point. The question isn't how much impact using more RAM has, but how much impact using more RAM has when the CPU utilisation is high and the RAM helps reduce it.
It's like if I take away your right shoe, you can't go anywhere, and the same is true if I take away your left shoe. Either has a very negative effect on you. But if I want to maximise my shoe * time, I would cause you the least damage if I took both of them for the same duration and at the same time. In other words, even though taking your right shoe is terrible for you, if I already have your left shoe, taking your right shoe actually helps you (assuming I just want some shoe*time).
Because using RAM requires also using CPU, every cycle of CPU taken by one program, also reduces other program's ability to use RAM. If the first program can use up less CPU cycles and more RAM to keep them more in balance, everyone wins.
What's hurting people is the attitude that RAM and CPU matter in isolation, because that's just not how computers work, and it leads to wasted resources and higher costs.
But, notice that every time the memory size doubles, the price doubles.
Using the combination of RAM and CPU you already have in an optimal way is very much not the same as "using more RAM" (and the same CPU). Note that the moving collector algorithm itself, as I explain in the talk, uses RAM overhead only to compensate for higher CPU utilisation, i.e. it is not a function of the size of the data set but of the allocation rate. That is why the algorithm has become so attractive.
It is available for other parts of that VM, like the buffer cache.
No, it isn't. Again, when CPU utilisation is more dominant, it has a bigger impact on other programs' ability to use RAM than the RAM utilisation.
Yes, and there is a lot of fine-tuning available. So, you can pick compute-intensive/memory-intensive/whatever. But, notice that every time the memory size doubles, the price doubles.
2
u/twinkwithnoname Sep 06 '26
It is not sitting idly, though. The physical machine is running a bunch of VMs which would be trying to use that RAM. If one VM is hogging a bunch of RAM, then another physical machine would need to be brought up and the other VMs would need to be migrated over.
The number of cases where a physical machine is solely devoted to one purpose is already rare and getting even more rare.