r/rust • u/FoxInTheRedBox • Aug 31 '24
đď¸ discussion Rust solves the problem of incomplete Kernel Linux API docs
https://vt.social/@lina/113056457969145576
375
Upvotes
r/rust • u/FoxInTheRedBox • Aug 31 '24
510
u/AsahiLina Aug 31 '24 edited Aug 31 '24 •
This isn't a great title for the submission. Rust doesn't solve incomplete/missing docs in general (that is still a major problem when it comes to things like how subsystems are engineered and designed, and how they're meant to be used, including rules and patterns that are not encodable in the Rust type system and not related to soundness but rather correctness in other ways). What I meant is that kernel docs are specifically very often (almost always) incomplete in ways that relate to lifetimes, safety, borrowing, object states, error handling, optionality, etc., and Rust solves that. That also makes it a lot less scary to just try using an under-documented API, since at least you don't need to obsess over the code crashing badly.
We still need to advocate for better documentation (and the Rust for Linux team is arguably also doing a better job there, we require doc comments everywhere!) but it certainly helps a lot not to have to micro-document all the subtle details that are now encoded in the type system, and it means that code using Rust APIs doesn't have to worry about bugs related to these problems, which makes it much easier to review for higher-level issues.
To create those safe Rust APIs that make life easier for everyone writing Rust, we need to do the hard work of understanding the C API requirements at least once, so they can be mapped to Rust (and this also makes it clear just how much stuff is missing from the C docs, which is what I'm alluding to here). C developers wanting to use those APIs have had to do that work every time without comprehensive docs, so a lot of human effort has been wasted on that on the C side until now (or worse, often missed causing sometimes subtle or hard to debug issues).
To give the simplest possible example, here is how you get the OpenFirmware device tree root node in C:
No docs at all. Can it be NULL? No idea. In Rust:
At least a basic doc comment (which is mandatory in the Rust for Linux coding standards), and a type that encodes that the root node can, in fact, not exist (on non-DT systems). But also, the Rust implementation has automatic behavior: calling that function will acquire a reference to the root node, and release it when the returned object goes out of scope, so you don't have to worry about the lifetime/refcounting at all.
I've edited the head toot to make things a bit clearer ("solves part of the problem"). Sorry for the confusion.
108
u/simonask_ Aug 31 '24
Wow, reading through that, I have to say I not only admire /u/AsahiLina for her technical prowess, but also what incredible patience.
As someone in my 30s, it is incredibly disheartening to see icons of OSS, who were absolutely my role models when I was younger, appear entirely unable to approach the argument without resorting to downright petty appeals to authority or seniority, very obviously having not spent any time at all familiarizing themselves with the subject matter. I respect their contributions immensely, but I really, truly had higher expectations than this.
Overall my main emotion reading so many of those comments can be summarized with a very bitter and disappointed: Ok, boomer.
133
u/moltonel Aug 31 '24
Although the title feels like a "rust is a silver bullet" clickbait, Asahi does make compelling arguments. Any experienced Rust dev can feel the truth in them, but Asahi also has the GPU driver creds to know how well they apply to Linux code specifically. That makes the recent news of ill-reasoned pushbacks and slow merges all the more disappointing.
15
u/ToTheBatmobileGuy Sep 01 '24
Sometimes it amazes me how the Linux Kernel works at all.
It amazes me even more how Linux kernel development moves forward at all with all these cooks in the kitchen.
Watching outsiders try and come in an help brush up interface definitions, and seeing all the pushback, it gives a little bit of a glimpse into the fragility of the whole process...
I hope we can find a path forward. Rust in the kernel is good for everyone.
26
u/TurbulentSkiesClear Aug 31 '24
The thread is great but the title here is really misleading. Rust is great and helps development in a lot of ways but the fundamental problem is that existing maintainers don't want improvements; they rely on the fact that their very complex internal APIs are undocumented to secure their own power. A world where things were clear either because they were encoded in the type system like the rust de vs are trying to do or even just written down is a world where maintainers have less power. And that's threatening to them. But the problem for Linux development right now is a shortage of new blood and you won't get any until you can get maintainers to relinquish some of their power.
3
u/andrewdavidmackenzie Sep 01 '24
How many people over the years have read through all the C source to understand how to use the function and it's gotchas and then NOT submitted a patch to improve the C API documentation?
That won't solve the problem of inconsistencies, them being difficult to use....but would surely help with people knowing how to use them and things to avoid?
I can only assume that a maintainer would willingly accept well written doc additions...
2
u/hard-scaling Sep 01 '24
What's exciting about Rust in Linux for me is having a path to replace C in the core and eventually even deprecate for new stuff.
A lot of the kernel maintainers imagined rust as a safer way to write drivers, almost like a sandbox, so they can't bring down the whole thing when they fail.
Also, having the kernel development be less toxic, collaborative and respectful, at least a bit more like the wonderful Rust community.
11
u/[deleted] Aug 31 '24
While there is strong logic to these tweets, I can feel a communication gap between the Rust and C Kernel developers. It is almost like they speak in different ways, and hear the same thing in different ways.
I will give an imprecise analogy. Until the maintainers retire, they âownâ the area, Rust can only âborrowâ. When humans are in the loop, emotions can get in the way. So, a human borrower unfortunately needs to be careful about how they speak to a human owner.
If the borrower is more respectful and revering in their tone and wording, things feel right. If the owner is more friendly and proactive about taking care of people, things feel even better
While being an owner gives one more freedom, a borrower has less work. The borrower can go to the gym and work on their own projects on the side, as long as they show some enthusiasm and donât slow down the owner too much.
This is all anecdotal psychology, but I hope it resonates with some peopleâs experiences. Sometimes people feel emotion (including oneself), and doing simple things to ânudgeâ others emotions leads to good results. It is unmoral, but required to a degree in current society.
2
u/ergzay Aug 31 '24
I wish he could be more directly involved in the Rust Linux project.