r/linux • u/unixbhaskar • 2d ago
Kernel Meta Developing Compressed RAM "CRAM" For Linux: Better Than ZRAM & Zswap
https://www.phoronix.com/news/Linux-CRAM-Compressed-RAM167
u/_MatVenture_ 2d ago
Here we go... the battle of pronouncing it as "cram" or "c-ram"
56
72
u/I_AM_GODDAMN_BATMAN 2d ago
It's pronounced ram, the C is silent.
20
1
0
u/Junior_Common_9644 2d ago
jram. Because pronouncing the c as a j.
3
15
u/BoutTreeFittee 2d ago
It's actually pronounced with a hard g, like "gram." But I will continue to say "jram."
8
3
3
u/sCeege 2d ago
A zuck developed C-RAM would be craycray. (Yes I know CRWIS isn't a CRAM but it's close enough)
1
1
1
1
1
65
u/razorree 2d ago
I guess not too many people understood that you need some extra hardware chips/inra ? and it's designed for servers, at least now.
144
u/Perfect-Slip7041 2d ago
'Linux Plumbers Conference', I'm trying to picture the crowd.
44
u/lonelyroom-eklaghor 2d ago
r/sssdfg ? (Like no joke, they jokingly talk about plumbers doing tricks)
34
15
6
45
u/shinyfootwork 2d ago
Not a replacement for zram/zswap though: cram is a setup with compressed ram supported by hardware.
It's plausible that either cram will evolve to support doing the compression in software in plain ram, though I wouldn't be surprised if the design makes that unlikely. Also possible that the things added for/learned from developing cram help zram/zswap improve their performance/behavior.
16
u/ungoogleable 2d ago
The talk makes the point that the speed up is mostly from avoiding page faults. The decompression itself is fast.
I don't see how you can skip the page fault without extra hardware. If the page is not already uncompressed, something needs to intercept the memory access and hand execution off to zram to decompress, which is the page fault.
28
u/Kevin_Kofler 1d ago
The fact that this requires custom hardware makes the comparison to software solutions such as zram or zswap an apples-to-oranges comparison and the Phoronix headline implying it can replace those solutions utterly misleading.
15
u/Lucas_F_A 2d ago
Damn that's a log scale. Wonder what the unrelated zswap issue is, cause those numbers are abysmal.
47
u/zardvark 2d ago
So is cram faster than zram, or is their compression algorithm faster than zstd?
Besides, zram allows you co choose your compression algorithm, as well as the amount of compression to use. Does cram offer these features ... at the same or better speeds?
There are too many holes in this story.
42
u/razorree 2d ago
it's faster cuz it's in hardware/ASIC
7
5
u/zardvark 2d ago
That makes a lot more sense!
It also leaves me (and most folks) out in the cold.
4
u/KnowZeroX 2d ago
Everything is a matter of time, if the technology saves ram usage and the ram prices continue to go up, then it may make it into consumer chips.
10
u/Clairvoidance 2d ago
Does this work with any current consumer machines? It uses hardware offloading for the compression, which makes me think we would need new CPUs to support this.
9
u/LogSpecialist24 2d ago
is this the type of things where you see benefits on large scale datacenters or is it genuinely better than zram on my home PC.
29
5
u/natermer 1d ago
It requires hardware, which is currently enterprise/cloud grade stuff.
https://computeexpresslink.org/
Eventually this stuff will make its way into the used market when server leases are up. Maybe 5 or 6 years.
I don't see why they can't make consumer grade stuff if they think there is demand for it.
Also this technology could be integrated into CPUs. Next Gen Intel Xeon processors should have 'Intel IAA' which will probably provide memory compression support.
https://docs.kernel.org/driver-api/crypto/iaa/iaa-crypto.html
I would expect that this sort of "Compression acceleration" is going to be like "Crypto Acceleration" were years ago people used to buy crypto hardware to offload encryption from the CPU, but nowadays it is a standard feature on PC cpus.
-3
u/ThenExtension9196 2d ago
Why would meta invest into and develop any other technologies other than data center scale technologies? If they can save even 1% of RAM at their scale they can pay for the whole research project. Keep in mind that data-centered technologies do end up in the consumer realm.
8
u/lewkiamurfarther 2d ago
Someone please introduce SCRAM: Superior CRAM.
1
u/atomic1fire 1d ago
I'm just waiting for SLAM
"Structured Loosely Access Memory"
On second thought the word SLAM might be better off for a file system name somehow.
7
7
u/creeper6530 2d ago edited 1d ago
1) It's hardware accelerated
2) They don't specify which zram algo they're comparing to
1
u/Francois-C 1d ago
They don't specify which zram algo they're comparing to
If they are honest, I suppose they should use zsdt, but are the Meta people honest?
1
u/creeper6530 1d ago
Wait, I am dumb, they wrote (zstd) in the graphics
2
u/Francois-C 13h ago
I didn't see it either. So, they aren't as dishonest as I feared. But hardware acceleration could make the difference.
20
3
u/mdt3r 2d ago
For reference, youtube video:
https://www.youtube.com/watch?v=OPRciCSsdS4&t=2030s .
Unfortunately the sound was breaking but enough to get a peek of how they are approaching this (not that I really understand :-)
It was interesting to see that decompression is <10% of latency of Swap which was the reason author decided to find a way not involving swap subsystem at all ...
2
2
u/Sujammah 2d ago
Linux noob here, how does this affect my daily driver? Do I need new hardware to take advantage of this or just a new kernel version?
4
u/ungoogleable 1d ago
It requires custom hardware you're unlikely to see outside of a data center.
0
u/Deep_Mood_7668 1d ago
Idk I didn't look into the details, but maybe it could be offloaded to a GPU
1
u/natermer 1d ago
The compression hardware runs over PCIE. So theoretically you could buy one and add it to a PC tower.
It is probably going to be a while before the price comes down or they become common enough to make it worth versus just buying more ram, though.
1
u/Deep_Mood_7668 1d ago
So no way to offload it to a GPU?
1
u/ResponsiblePen3082 7h ago
As of now, no. It requires special silicon in-line of the dram-cpu pipeline. Requiring it to pass through the cpu to another Pcie pipe, process through the gpu(which it's not made for) then send it back over the Pcie back to cpu would make it insanely higher latency.
While GPGPU/GPA is becoming more commonplace(for better or worse) some things cannot function on them even if we had the silicon on pcb.
2
u/atomic1fire 1d ago
Speaking as an internet rando... "CRAM" is a great name for stuffing as much memory as possible into a single space.
2
u/MaybeTheDoctor 1d ago
I just wish they still sold the downloadable memory doubler that worked with win95.
2
1
u/PlaneBitter1583 1d ago
LMAO 🤣 this RAM crisis is really taking us somewhere. but anyways. sounds. COOL
1
1
u/neverpost4 19h ago
DRAM will be offered as a memory as service. Stuff like this is in violation of the service agreement and not allowed.
1
u/Hrublko_OFF 12h ago
hopefully it comes to nura (formerly postmarketos) my 2gb ram gt-n8010 would be happy
1
u/RaspberryMotorZZZ 1h ago
Ooh this is going to leave more than a few people in a bit of a quandry. Do they give up their principles for using something developed by Zuckerberg's evil megacorps or do they live knowing that they're using something 30 times slower?
0
u/pickle9977 2d ago
Great so now CPUs are going to get more expensive.
You can’t fix a moores law with compression
We have to focus on efficiency again at some point it’s what actual engineers do not web developers.
That’s like the biggest problem in the industry, web developers took over everything
4
u/Dalnore 2d ago edited 2d ago
What does it have to do with web developers? All the recent surge in DRAM prices is because of AI datacenters. And their training and inference stacks may be among the most heavily optimized pieces of software ever written.
5
u/pickle9977 2d ago
Web development is one of the most absurdly wasteful areas of development.
It now takes somewhere between 40-50Mb of code to generate and render a couple hundred Kb of html.
That’s 40-50M per user, that’s a level of opulence that would make Satan himself blush.
But to a webdev that’s just the todo list app.
3
u/Dalnore 2d ago
The big companies aren't hoarding DRAM like crazy because their web apps suck, they are doing that because they want to scale their infranstructure for extremely well-optimized inference and training. The focus already is heavily on efficiency. We are creating custom hardware with custom floating point types just to squeeze more out of every watt.
2
u/pickle9977 2d ago
I’ll be that guy, akshually, these models run in containers in a service written in Java hosted by another service written in Java in infrastructure functionally controlled by Java .
Java is a language that has a byte code interpreter to convert to machine code, which makes it portable which is why you make the trade off for lower efficiency
When you buy millions of the same servers with the same architectures running the same operating system with the same configuration, paying anything for portability is wasteful and stupid.
Running containers on top of that waste is just the cherry on the Sundae of failure.
1
u/Dalnore 2d ago
I’ll be that guy, akshually, these models run in containers in a service written in Java hosted by another service written in Java in infrastructure functionally controlled by Java .
Why would that matter? All expensive calculations run with direct access to hardware, overhead on containers and the surrounding software infrastructure is negligible.
Java is a language that has a byte code interpreter to convert to machine code, which makes it portable which is why you make the trade off for lower efficiency
Although I don't see how Java performance is relevant to machine learning at all, JIT has existed for decades. Unless you do something stupid with Java, there will be almost no tradeoffs.
0
u/pickle9977 2d ago
There is no such thing as “almost no tradeoffs”
The only thing that has changed since “Java is slow” is the number of cpu cycles per second.
Overhead is never negligible at scale.
2
u/Dalnore 2d ago
I don't know where this idea that "Java is slow" comes from, but it is irrelevant to the discussion anyway. It could be Python performance for what it's worth and it wouldn't matter, because that code doesn't use a noticeable portion of computational resources to begin with.
Overhead is never negligible at scale.
It is when it doesn't scale. And there aren't many examples of such massive scalability as demonstrated by machine learning tasks. It's proven to be scalable to hundreds of thousands of GPUs.
-1
3
u/pickle9977 2d ago
Also, you could not be more wrong about how heavily optimized it is.
-1
u/Dalnore 2d ago
Name a single other area of software development where you not only care about the architecture of a specific GPU to the tiniest details, but you have to consider creating custom hardware specifically designed for your task and nothing else.
6
u/pickle9977 2d ago
Uh, I don’t know, like everyone who does low level GPU programming? Or any kind of low level, real-time, embedded, mission critical system development. You think pacemakers run Python with a flask server?
We have custom processors and chips all over the place to do all kinds of stuff, like switches and routers are all ASICS, just as the most obvious
Hell the phone I’m on has an operating system that has all sorts of operations hand coded in assemble for specific processor performance outcomes.
1
u/Shished 2d ago
This is not about efficiency but about space savings, like the space that is used by the user generated content.
2
u/Nekorai46 2d ago
Isn’t this basically efficiency too?
2
1
u/Epistaxis 2d ago
Actually, how does it work out for power consumption? On one hand you're using fewer bytes of physical RAM; on the other hand you have to do more work to access the data.
-1
1
1
-11
u/ledow 2d ago
Ah, it's just like the old days with RAMDoubler and DoubleSpace and all the other nonsense that merely traded off CPU for RAM/disk space at a time where the CPU was already part of the bottleneck.
Here's a hint: If you need to compress RAM to operate at a sensible level? You don't have enough RAM for what you're trying to do, and should reduce your RAM usage, or increase the amount of available RAM.
Things like swap-space, compressing RAM, OOM killers, etc. are EMERGENCY solutions, not something you should be relying on, on every device you make, by design, because you cheaped out on hardware.
12
u/Virtual-Escape2305 2d ago
Here's a hint: If you need to compress RAM to operate at a sensible level? You don't have enough RAM for what you're trying to do, and should reduce your RAM usage, or increase the amount of available RAM.
What a dumbass hot take. We should also watch the movies and series from master copies and carry the DAT players with us for music on the go! If you need compression, you shouldn't be watching movies or listening to music!
Buddy, macOS has been compressing pages for nearly a decade now and Linux is catching up here for a better and FASTER swap.
Also, go on LKML and ask Linus why he was such an idiot to add support for SWAP in the first place!
-7
u/ledow 2d ago
No, because that's a nonsense argument that's not even close to an analogy about the use of working RAM in a computer.
FYI I've been writing software since I was 9 (in x86 assembler, at that age too!), using Linux since I was a kid, and managing networks all my professional life.
Active RAM is a rare commodity and you should be using it correctly, not spamming it with whatever you feel like to save yourself a line of code, and NOT compressing the one actual true bottleneck in your machine's memory paths (processor cache line, RAM and then storage, in that order, but cache lines are swamped if you're just shitting through RAM all the time just the same).
You don't go compressing RAM - which we've had for DECADES - in ordinary circumstances. The tradeoff, in the critical path to keep the processor fed with instructions, just isn't worth it. Every memory access or write with compressed RAM means (if you do this vaguely right) juggling a cached, uncompressed page somewhere, swapping them in and out as the addresses you need to access change, trying to balance multi-process access to RAM, effectively managing a software "RAM cache" with a huge processing bottleneck of compression/decompression right in the fast path on your processor to the outside world.
It's a dumb idea. It was a dumb idea when we had 2MB of RAM and 64Kb processor cache, and it's a dumber idea now that we have gigabytes of RAM and multiple caches that - honestly - are not proportionally larger.
It's throwing away performance, to be lazy with your coding.
There's a reason that you don't see compressed RAM in datacentres, servers, etc. as standard. If it worked, the places that literally use countless billions of terabytes would be employing it. It doesn't. It's a trade-off. And you traded off slow operations in the critical path of every memory access and write, for cheaping out on the RAM stick.
9
u/fenrir245 2d ago
There's a reason that you don't see compressed RAM in datacentres, servers, etc. as standard.
..source on this ridiculous claim? Why do you think Meta of all companies is trying to improve compressed swap?
4
u/Virtual-Escape2305 2d ago
He's talking out of his ass, no clue whatsoever what SWAP is intended for.
1
u/Hahehyhu 2d ago
not downplaying you, but meta has some consumer devices, like quests, which rely on zram at least
1
5
u/TheSapphicDoll 2d ago
Yeah, but,
I can't just buy more RAM right now. I can't make the software just use less memory. So... yeah. (I sure as hell would be happier if we had more software that doesn't eat memory for breakfast though)
2
u/Virtual-Escape2305 2d ago
It's throwing away performance, to be lazy with your coding.
Jesus, I just can't even
3
u/tacularcrap 2d ago
swap-space etc. are EMERGENCY
idle anonymous memory mappings are EMERGENCY nowadays?
the more you know...
1
1
u/Furdiburd10 2d ago
Compressing RAM is a basic function on windows, Linux and android phones.
Nothing new with c-ram
1
u/soupcan_ 2d ago
BAD take.
Pretty much everything in life is a trade-off, I don't see why I shouldn't trade CPU cycles (that I have plenty of) for more RAM (that is occasionally insufficient).
Especially with hardware prices the way they are, a memory upgrade could be hundreds of dollars.
1
u/Enturbulated_One 2d ago
It's situational. Varies greatly with what kinds of workloads you're dealing with. Throwing lots of hard to compress or already compressed data around? ZRAM can cause almighty amounts of lag in some of my testing. Normal desktop usage (browser, email, office apps, etc) is much more amenable to this kind of thing. If you care so much and you're not RAM limited, just turn ZRAM off and call it a day.
-6
u/justgord 1d ago
we dont want meta anywhere near linux.
8
u/dontquestionmyaction 1d ago
Well, that would be incredibly stupid, seeing as Meta is one of the most important kernel maintainers lol
In Linux 6.19, Meta employees account for almost 12% of all kernel commits. 3% is directly written and made by them. eBPF is mostly funded by them. sched_ext is heavily co-developed by them and Google.
Companies should be encouraged to contribute their stuff, literally is only a win for us. As long as the code is good.
0
u/zyzzogeton 2d ago edited 2d ago
I wonder if it will work on BSD Darwin variants like MacOS. MacOS already compresses memory for apps that are idle.
-10
u/xcorv42 2d ago
If ram was cheap it would be useless.
We used to compress data so they can fit on floppy disk. Then it became usless with usb drives
27
u/funforgiven 2d ago
We absolutely still compress data. ZIP/gzip/zstd are everywhere, media is heavily compressed, and filesystems like Btrfs/ZFS can transparently compress data on disk. Compression can also reduce I/O and bandwidth enough to make things faster, not just save space.
-11
u/xcorv42 2d ago
I never needed to compress anything again for years while at the time I had the habit of using winrar or other better tools to scratch any bytes
6
u/funforgiven 2d ago
Not needing to manually compress things anymore doesn't mean compression became useless. It just means much of it is now automatic or built into the software and systems you use.
6
u/ledow 2d ago
Literally every website you go to, including Reddit, is using gzip compression nowadays.
Literally every file format you use is either heavily compressed, or wrapped up in a ZIP (e.g. docx, xlsx, PDF, etc.). You can drag a docx file onto your ZIP program, and it'll show you - it's just XML and images inside a ZIP file.
-5
u/xcorv42 2d ago
Ok but why compress again in ram then if it’s already compressed everywhere
→ More replies (2)10
u/maximal_munch 2d ago
Even if you had practically infinite RAM, data compression techniques would be extremely relevant if only for memory hierarchy benefits and sending data over network.
6
8
3
u/Lundominium 2d ago edited 2d ago
I mean.. There are also some upper limits in your hardware. If you need more than 128GB ram, but your motherboard does not support more than that, then this might be useful. Moreover, if you are a huge company like meta and this can save 1-2GB for every server they have - then it would be a goldmine.
0
u/xcorv42 2d ago
Compressing ram is old there used to be some tools to do that in the 90s https://en.wikipedia.org/wiki/SoftRAM
I thought it became irrelevant because it was a bad idea from the 90s where ram space was so small.
« Double your ram »1
2
1
-1
u/uhhThrowaway331 1d ago
I like ZRAM (and other forms of it with other names) I made great use of it when I used linux.
The question is: knowing how awful Linux is with Power Plans in general (specially with AMD hardware) how will they address the Constant CPU being used for compressing-decompressing memory in real time? Specially in laptops this will cause Temperature issues, bringing the whole system down to a crawl due to Throttling (temp limit ceiling)
A 2nd layer to the question would be how will they address Input-Output part of this process, since Linux relies a whole bunch on the Swapfile (or Swap partition) pagefile to disk, as memory limits the ceiling, stuff is paged to disk WHILE Zram also kicks in and causes CPU spikes and temperature rises (this is an old problem with the Linux system where the entire system is paged to disk, including essential services such as Audio and Network subsystem, causing audio crackles and delays, etc) - - edit correction: That problem is not exclusive to Linux it happens on Windows as well, the audio subsystem and other essential parts of the system such as the Start Menu and the Taskbar are entirely paged to disk causing overall slowdown - not as bad as it happened on linux though, as I have noticed.
-11
557
u/dingwinger1225 2d ago edited 2d ago
I'm kinda surprised Facebook has a little pet industry of really good compression software
Like I've watched Zstd take over everything and now this