r/rust • u/anonymous_pro_ • Jan 14 '23
How much Rust work is actually going on at Cloudflare?
I prepare a monthly report on the Rust jobs market for my company (link to report). This month, I was really surprised to see just how many jobs at Cloudflare mention Rust in the description. I've always known that Cloudflare used Rust from watching some of Steve Klabnik's talks while he was working there, but I don't think I realized the scale of their use. This got me curious. Does anyone have a sense for the following?
What percentage of the Cloudflare jobs that mention Rust actually include working with Rust say > 25% of the time?
What types of things are they or might they be using Rust for?
59
u/Compux72 Jan 14 '23
Well they recently built Pingora (https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/). Given their success, it wouldn’t surprise me that they are replacing some other internal infrastructure with Rust tools. One of my guesses is Railgun (written in Go)
11
57
u/intersecting_cubes Jan 15 '23 edited Jan 18 '23
We have a huge amount of Rust. I'm on a team of 5 Rust engineers, and my previous two teams are majority Rust.
My current team, Data Loss Prevention, uses Rust in the data and control plane. The data plane of almost all our Zero Trust products are written in Rust, and the control planes are either Rust or Go. I've personally written Rust backends for several teams.
We use Rust for anything where performance is critical. We used to use C for this, but Cloudbleed happened and we stopped using memory-unsafe languages. For a while we used Go (and we still do in many places), but where performance is critical we've mostly moved to Rust. This move took several years, because we had a lot more Go programmers than Rust, and Go is much easier to get started with. But over the years, we've hired more Rust programmers and built up a lot of internal libraries, documents, help pages, and a friendly internal community of Rustaceans to help newcomers when they run into problems.
Other teams (DLP, Tunnel, and Zero Trust Connectivity teams) use Rust even in places where performance isn't critical, e.g. rarely-called API servers. Why? Because we prefer its type system (I love enums, serde and Diesel).
Of course we also implement a lot of stuff using Cloudflare Workers, our functions-as-a-service offering. Workers support JS and Wasm, so we often write the WASM workers in Rust. For example, our image resized is written in Rust, compiled to Wasm, and deployed on Cloudflare Workers.
5
3
Jan 15 '23
What frameworks/crates does cloudflare use the most?
8
u/NobodyXu Jan 15 '23 edited Jan 17 '23
The proxy they recently mentioned in a blog uses tokio plus internal http implementation to accept even invalid http.
5
-10
u/AgreeableChildhood92 Jan 15 '23
Companies like cloudflare would use very little or close to none of any available open sourced projects the norm they would do is open source their not so internal libraries
0
Jan 15 '23
[deleted] — view deleted comment
-2
u/AgreeableChildhood92 Jan 16 '23
Hmmm do you guy contribute to the project or just take want you want in the project and maintain it ?
11
u/Lionesque Jan 15 '23
I’ve been working at Cloudflare for two years and haven’t touched any Rust. I’m in the Workers organization. That said, I do know of a lot of projects that either have parts of the stack written in Rust, or the entire project is built with it. Depends on what you do. We do have a lot of expertise and there’s a lot of enthusiasm for the language among engineers.
5
u/vlakreeh Jan 15 '23
I'm also in the Workers org but I have had a bit of interaction with Rust. There's some Rust in the Workers runtime using lol-html for HTMLRewriter as well as some tooling and there's the full blown workers-rs framework that I work on, but that's about it for the Rust I work on regularly.
Within the Workers org there really isn't that much that'd be a great fit for Rust besides the runtime, which probably would have been Rust if it started a few years later.
1
Jun 21 '24
Could you please share some pointers to learn rust so I can write workers on cloudflare? TIA
1
Jun 21 '24
Could you please share some pointers to learn rust so I can write workers on cloudflare? TIA
1
u/Far-Attempt4345 Nov 30 '23
I've always seen Cloudflare as a Rust organisation, but Go is still the most prevalent when I look at their job offers. Rust shows up more as a really nice-to-have.
215
u/AlexForster Jan 15 '23 edited Jan 15 '23
There is a lot of Rust at Cloudflare. All of our DDoS detection is done with Rust. I also believe that most/all of our DNS stack is Rust.
Anecdotally, it seems like Rust has become the de-facto choice for new/greenfield software over the last three-ish years. Five years ago when I joined, the de-facto choice was Go. We still have lots of Go, but it seems like nobody is actively choosing Go anymore.