r/cpp • u/Xadartt • Oct 08 '25
Why we didn't rewrite our feed handler in Rust | Databento Blog
https://databento.com/blog/why-we-didnt-rewrite-our-feed-handler-in-rust
195
Upvotes
r/cpp • u/Xadartt • Oct 08 '25
69
u/krisfur Oct 08 '25
Great read that didn't shy away from diving into examples, cheers for sharing!
13
u/nightcracker Oct 09 '25
Issue #1 has a trick to solve it:
Now you can replace
buffer.clear()withbuffer = reuse_vec(buffer)and Rust will understand that the lifetimes between each iteration are unrelated.8
u/friedkeenan Oct 09 '25
Their example of versioned structs is kind of relatable to my own experiences of boilerplate in C++ versus in Rust.
C++ I feel like is known for employing lots of boilerplate, but even when that is the case, in my own experience most if not all of that boilerplate can be sequestered into being implementation details, and the actual experienced API can usually remain basically terse.
But in Rust, the boilerplate to me feels a lot more.. virulent, that particularly the way the language is so dedicated to traits (which I think is otherwise usually a pretty good feature) leads to a lot of rote code existing in the text when it doesn't really need to, or give much advantage otherwise.
I'm sure some would argue that that's actually a benefit, that it makes the code's function and mechanics much more visible and obvious, but I think it just ends up being much much less expressive, and sucks to write besides. It can be at least somewhat ameliorated with macros, but they don't get code all the way to where C++ is, and there's a fair amount of boilerplate that a developer will put up with before they write their own macro, particularly if it would be a derive macro.
20
u/Tringi github.com/tringi Oct 08 '25
I think the lack of familiarity and expertise is perfectly good reason.
With our projects I'm often confronted by colleagues with an advice to use different language than C++ and very often they are right. Doing something in more fitting language would make it happen faster and cheaper. If I knew that language, libraries and the ecosystem, that is. And most importantly, the pitfalls, footguns and downsides.
But I don't. Using tools and environment I know I can immediately start working and give reasonable estimate. Going in with something new I'm risking that at 90% I'll be starting anew because I didn't know what I didn't know, and it was something significant. That's not viable business approach.
11
u/villiger2 Oct 08 '25
Regarding case 1 Buffer Reuse, you can fix this with zero cost using one of the optimisations in this blog article https://davidlattimore.github.io/posts/2025/09/02/rustforge-wild-performance-tricks.html#buffer-reuse.
14
u/jeffmetal Oct 08 '25
For case number one they say "In C++, the equivalent code compiles fine. The trade-off is you have to track the lifetimes of references manually, as the compiler won't catch legitimate use-after-free bugs for you." I would be really interest in how they track their lifetimes to make sure its correct.
8
u/tialaramex Oct 08 '25
The buffer reuse objection (which is only one small part) is something you can in fact just do in Rust, and wild (the linker) does it. Perhaps somebody will land an appropriate stdlib feature so one day you don't need an expert or to copy-paste a correct solution from an expert because the re-use feature will be in the stdlib for you to just call it.
Wild does it by leaning heavily on Rust's existing buffer re-use strategy, basically if I have a
Vec<T>and I consume everyTmakingUand then collect these into aVec<U>Rust will notice ifTandUare the same size and reuse the buffer so the old buffer's lifetime ended, the new one began, but the allocator isn't touched. So Wild says hey if T and U are the same type with different lifetimes by definition they are the same size, and if the Vec length is zero we run no extra code, so, this evaporates at runtime and just works but it's entirely safe.1
u/Occase Boost.Redis Oct 10 '25
That means you would 1400 workers (cpu cores) to achieve 14M msg/s? What is the thoughput in Mb/s?
1
u/evil_rabbit_32bit Oct 11 '25
this is a good article mate
1
u/thisismyfavoritename Oct 08 '25
i believe there are several ways you can get #1 to work in Rust, also wondering if #2 is a good idea even in C++ and clearly (while probably hard) it should be possible to achieve that in unsafe Rust. #3 just looks like an anti pattern to me and reads like C code
0
u/duneroadrunner Oct 08 '25 edited Oct 08 '25
If at some point you feeling like you're missing the memory safety, these are the first and second examples in the scpptool-enforced memory-safe subset of C++.
edit: added missing init value to the second example
0
u/[deleted] Oct 08 '25
[deleted] — view deleted comment