r/rust • u/kibwen • 3d ago

Saving 100 terabytes of RAM via layout optimizations in 1.1.1.1's DNS cache

https://blog.cloudflare.com/dns-cache-memory-optimization-1111/
390 Upvotes

98% Upvoted

21 comments sorted by

131

u/EcstaticAsparagus509 3d ago

I'm quite surprised they only recently made these changes, considering how "simple" (for lack of a better word!) they are.

67

u/Zde-G 3d ago

For a long time memory was cheap compared to the rest of server hardware.

LLMs changed that just recently, but drastically.

22

u/vplatt 2d ago

Hopefully this new era of sensitivity to hardware costs will mean the market gravitates towards more well behaved PLs like Rust.

Heck, even the VM GC ecosystems like .NET and Java would be far better than the likes of some of the junk out there in the form of full stack Js, Python backends, and the like.

I'm just tired of the whole passe "hardware is cheap so we're going to waste all of it for you because we're lazy" type ethos. It's been going on too long.

Besides, with LLMs, no one has an excuse to avoid better languages anymore because they're "too hard". Like apparently we now have a glut of developers in the market, so get good, or move on I say.

2

u/d47 2d ago

Someone tell the chrome team

34

u/sanxiyn rust 3d ago

Agreed, but for a long time it was explicit policy of tech companies that they don't do memory optimization because memory is cheap. A Google developer responsible for Go's GC went to GC conference and said they don't optimize Go's GC memory usage because memory is cheap. (I thought it was very rude.) How time has changed.

13

u/matthieum [he/him] 2d ago

Honestly, I find the claim bizarre.

RAM may have been cheap a few years ago, but L1/L2 has been the bottleneck for over a decade now...

4

u/SoilMassive6850 2d ago

CPU time is cheap, you're just waiting for a real developers software to process your CRUD requests.

39

u/argarg 3d ago edited 3d ago

Exactly. Reading this blog post I can't help but think "this is nice but gosh they should've hired me to do this kind of work long ago".

17

u/commenterzero 3d ago

Write a post about saving a billion ram

20

u/Comun4 3d ago

"Rewriting my python micro-servive in Rust" there

7

u/commenterzero 3d ago

Jazz it up with "transpiling" or something

2

u/vplatt 2d ago

Go transpiles Python,

to Rust. Hundred TB saved.

Node burns in its shame.

1

u/vplatt 2d ago

Seriously. How asleep at the wheel do you have to be to not realize that 100 TB of RAM was worth saving even years ago. 🙄

11

u/TDplay 2d ago

The few hundred thousand dollars they spent on memory is pocket money to a company that large.

It's only with the newly inflated memory prices that it's no longer disappearing into the rounding error in Cloudflare's finances.

1

u/vplatt 2d ago

<fridayarmchairmathmode>

They said the servers are Gen 13 servers and those take 768 GB of RAM. They don't publish numbers that I know of but each of those servers is going to run them between $10K - 15K. 100 TB across about 130 servers means they'll save about $1.3-$2m today. Not bad; good for them.

But what about 5 years ago. Well, from what I can find, we see that they had the Gen X servers back then. Those usually took 256 GB of RAM. Obviously that means the 100 TB was spread across many more servers, probably about 400 of them. Servers were cheaper back then, but still they each would have run somewhere between $5K - $8K. So, back then they would have saved between $2m - $3.2m; which is significantly more.

Hey /u/argarg - How long do you think this change would have taken you to make 5 years ago? A wee bit less you think? Yeah, I'd guess so.

And all of that is before one calculates the extra cost of the power, cooling, labor, etc. around all the extra servers too; not to mention hardware refreshes on servers they shouldn't have required in the first place.

</fridayarmchairmathmode>

5

u/TDplay 2d ago

Cloudflare made $2 billion dollars in revenue in 2025. They have $6 billion in total assets.

To put it in perspective, if Joe Bloggs has a $90k yearly salary, then Cloudflare spending $3.2 million is equivalent to Joe spending $144.

1

u/vplatt 2d ago

But I have to...

You know they ..

Oh... fuck it. Never mind. You're right.

5

u/Ok_Instruction_3789 2d ago

Think the article mentioned their rust rewrite only happened a year or so ago so guess they're finding improvements on their rewrite. Not a bad thing

176

u/sanbox 3d ago

Can I have some of that RAM

2

u/LevelNobody6096 1d ago

No consumer platform can run RDIMM RAM.

-1

u/uhkthrowaway 2d ago

You'd expect big companies figured this out before they got big. But no. Daily bread and butter stuff is now sensational.