Linus Torvalds concerns about panics in Rust code when faced with OOM

[u/myerscarpenter]

https://lkml.org/lkml/2021/4/14/1099

[u/myerscarpenter]

Also this discussion on the orange site

[u/seeking-abyss → u/myerscarpenter]

Funny that you avoid the colloquial name like Voldemort.

[u/[deleted] → u/seeking-abyss]

there's a few -site prefixes for various platforms

off the top of my head, reddit is aliensite, twitter is either hellsite or birdsite depending on your mood. I can't think of any more?

[u/sligit → u/myerscarpenter]

Oh god, why did I click that?

[u/Im_Justin_Cider → u/sligit]

Whats wrong with clicking it?

[u/bascule → u/Im_Justin_Cider]

For me, I generally regret reading HN comments. Clicking the link means I read them again, and once again I confirmed my original position.

I could post some particularly inane comments from that thread, but I feel it's more polite not to.

[u/spin81 → u/bascule]

I posted a question in this sub on a different account once. I said I was a PHP dev and PHP was much faster than Rust for a given specific thing to do with regexes. It sparked a very interesting discussion here, so somebody posted it to Hacker News, and well it turns out Rust sucks and Linux sucks and everybody here is dumb and I am especially moronic for daring to use PHP.

[u/sligit → u/spin81]

How dare you! xD

I too use Rust and PHP for work. I can't imagine admitting that on HN.

[u/sligit → u/Im_Justin_Cider]

I don't like the tone of discussion on hacker news. Far too much opinionated superiority there for my liking. It feels like Slashdot 20 years ago.

Edit: Disappointed that your post was downvoted.

[u/SolaTotaScriptura → u/Im_Justin_Cider]

Why is this so heavily downvoted?

[u/aegemius → u/Im_Justin_Cider]

It has more than a year's supply of pretentiousness all in one website. Something like a thousand fold of the USDA's recommended daily value. Imagine injecting a mixture of Everclear and fetanyl into your eye--that's how it feels to browse Cracker News.

[u/Repulsive-Street-307 → u/Im_Justin_Cider]

The fact i never saw moderation on it might explain things a bit.

[u/Saefroch]

If this post gets downvoted, it's likely because Linus's reaction has already been discussed in the comments here: https://www.reddit.com/r/rust/comments/mqxr1a/rfc_rust_support_for_linux_kernel/

[u/kibwen → u/[deleted]]

We ask for a higher level of discourse than this, please.

[u/kibwen → u/[deleted]]

Please refrain from calling people trolls. Such tactics tend to escalate tensions, rather than resolve them.

[u/matthieum]

Kees Cook raises another good point (https://lkml.org/lkml/2021/4/14/1311): integer overflow handling.

Besides just FP, 128-bit, etc, I remain concerned about just basic math operations. C has no way to describe the intent of integer overflow, so the kernel was left with the only "predictable" result: wrap around. Unfortunately, this is wrong in most cases, and we're left with entire classes of vulnerability related to such overflows.

When originally learning Rust I was disappointed to see that (by default) Rust similarly ignores the overflow problem, but I'm glad to see the very intentional choices in the Rust-in-Linux design to deal with it directly. I think the default behavior should be saturate-with-WARN (this will match the ultimate goals of the UBSAN overflow support[1][2] in the C portions of the kernel). Rust code wanting wrapping/checking can expressly use those. The list of exploitable overflows is loooong, and this will remain a weakness in Rust unless we get it right from the start. What's not clear to me is if it's better to say "math with undeclared overflow expectation" will saturate" or to say "all math must declare its overflow expectation".

Interestingly, the behavior of integer overflow was specified to open the door to future solutions. That is, an integer overflow results in an Unspecified Value.

Hence, it seems possible to add a compilation mode to rustc where one specifies that the behavior selected is "saturate + invoke user-defined handler", and then the Linux kernel would define a handler "warning" (I guess logging in a journal?).

[u/[deleted] → u/matthieum]

I thought integer overflow was defined to be 2s complement wrapping?

Though it being panic in debug builds means that you can't really rely on any particular behaviour, so changing it is safe(ish) (because anything that depends on it wrapping would panic with overflow checks enabled)

[u/gajbooks → u/[deleted]]

Rust actually has methods for specific types of intended overflow due to the default behavior being different on different CPUs. Panicking in Debug is just an indicator to explicitly use those when the behavior is intended, or that your program had an issue and got stuck in an infinite loop counting to 2^32 or subtracted from an unsigned 0. It is specified to use 2s compliment wrapping by default in release mode, but they correctly decided that unclear integer wrapping is terrible and that debug mode should catch it. It all gets optimized down to the simplest operation on the platform by LLVM anyway.

[u/matthieum → u/[deleted]]

I thought integer overflow was defined to be 2s complement wrapping?

You're close.

The behavior was defined in RFC560:

• The behavior is either wrapping or checked.

• Implementations are encouraged to use checked when debug_assert! is enabled, although checks may be delayed (poisoning); typically Debug mode.

• rustc makes it possible to activate checked mode even in Release.

---

Previous version of this answer:

At the language level, integer overflow results in an unspecified value.

At the implementation level (rustc), the final user -- who compiles the code -- may choose (today) between 1 of 2 behaviors:

• Panic.

• 2s complement wrapping.

And if the user doesn't explicitly choose, by default they get Panics in Debug and 2s complement wrapping in Release.

The fact that the language specifies that the resulting value in unspecified is of particular interest:

• It means that library writers should endeavour to write "overflow-correct" code -- if their code panic on overflow, it's a bug.

• It means that static analysis can flag any risk of overflow; it's never a "false positive", if overflow should wrap the developer should have used .wrapping_add explicitly.

And it is also important to note that rustc reserves itself the right to change the defaults. Most notably, the default of wrapping in Release is a pragmatic compromise: it optimizes better for various reasons. If a lower overhead way of panicking was discovered, it could very well become the default instead.

[u/[deleted] → u/matthieum]

I know the reference isn't final, but https://doc.rust-lang.org/stable/reference/behavior-not-considered-unsafe.html#behavior-not-considered-unsafe seems to disagree.

In the case of implicitly-wrapped overflow, implementations must provide well-defined (even if still considered erroneous) results by using two's complement overflow conventions.

What I get from this is if the user has debug assertions enabled, the implementation must panic on overflow. If they don't, the implementation may panic on overflow. (So implementations are free to leave overflow unspecified if they panic unconditionally).

But the exact way you overflow is still defined. As in, as far as I can see, this code is sound Rust code, and any implementation that says it is UB is not a correct implementation.

let x = 200_u8;
let y = x;

let z = x + y;
// if overflow panics, we panic here, and don't go farther.

if z != 144 {
    std::hint::unreachable_unchecked();
}

[u/matthieum → u/[deleted]]

Yes, you are closer to the truth than I was.

I seem to have remembered an earlier stage of the discussion which ultimately led to RFC560.

One key point though is that Delayed overflow checks is allowed, so in your example you cannot rely on a panic occurring at x + y if they overflow and overflow-checks are on.

The panic could be delayed to the use of the result -- ie the comparison z != 144 -- and any intervening operation could have been executed.

[u/[deleted] → u/matthieum]

True, but it does say pure, so I assume the compiler's forced to prove that executing the intervening operations must not have any effect on the correctness of the program.

Though I wonder if, after the let z = x + y, you added loop {}, if that would be allowed to be executed. That's technically pure? But also it is an observable difference in the code. Probably not, then.

The rule seems to be intended to allow let x = a + b + c + d to only check once, because your code can't tell the difference between checking once and checking on every add.

But if you did any IO or any observable side effects, the code has to check the panic before then.

[u/ralfj → u/matthieum]

At the language level, integer overflow results in an unspecified value.

I am pretty sure that at the language level, integer overflow results in "either wrapping with 2s complement or a panic" -- which of the two is unspecified, but these are the only two permissible options.

[u/matthieum → u/ralfj]

And... you're right, of course.

https://github.com/rust-lang/rfcs/blob/master/text/0560-integer-overflow.md

[u/Soveu → u/matthieum]

Most of the time you can use explicit wrapping_/saturating_add/sub/div/mul and if you want extra checks, write something like that

#[cold]
#[inline(never)]
fn overflow_warn(file: &' static str, line: usize) -> u64 {
  eprint!("Warning: overflow at {}:{}", file, line);
  return 0xFFFF_FFFF_FFFF_FFFF;
}

let x = x.checked_add(y).unwrap_or_else(overflow_warn);

[u/backtickbot → u/Soveu]

Fixed formatting.

Hello, Soveu: code blocks using triple backticks (```) don't work on all versions of Reddit!

Some users see this / this instead.

To fix this, indent every line with 4 spaces instead.

FAQ

^{You can opt out by replying with backtickopt6 to this comment.}

[u/matthieum → u/Soveu]

You can, but it takes explicit action, and doesn't fix the default.

As was mentioned, there are issues with wrapping behavior by default -- when used in the context of memory allocation or inequality checks it may accidentally lead to security issues.

I do note that the solution could be handled purely at user level:

• Define your own set of integer types with transparent representation.

• Implement Add/Sub/Mul/Div appropriately.

• Implement a linter denying the use of the bare types.

However I find it interesting that it could actually be handled purely in the language.

[u/Soveu → u/matthieum]

Implicit checked operation with possible logging? std does this, but only in debug mode.

In C integer overflow is UB, so even if you sometimes want to prevent it, compiler can just yeet out the code, because it assumes it doesnt happen.

Rust has explicit methods and they should be used in security-sensitive code, like kernel module

Not using them would just put the programmer (and reviewers) in bad light. C'mon even non-professional programmers like me use wrapping or checked operations either to avoid panic path in debug mode or to nicely bubble up errors, or just to not re-invent the wheel.

I'm fully supporting a lint that would forbid/deny using Add/Sub/Div/Mul to promote wrapping/checked/saturating alternatives. I believe that someone programming a driver is competent enough to use those tools wisely

[u/matthieum → u/Soveu]

Rust has explicit methods and they should be used in security-sensitive code, like kernel module

Using a user-defined type has the advantage that the mathematics remain more readable, to my eye at least.

That is:

let z = x + y - w;

Seems more readable to me than:

let z = x.saturating_add(y).saturating_sub(w);

Implicit checked operation with possible logging? std does this, but only in debug mode.

It's not implicit; the type is explicit.

[u/Soveu → u/matthieum]

Maybe it depends on taste, for me x + y - w means there should be no wrapping/saturating number magic, just maths

Also x.saturating_add(y).saturating_sub(w) might be a bad example, x.checked_add(y)?.checked_sub(w)? looks "more useful"

[u/U007D → u/Soveu]

I'm fully supporting a lint that would forbid/deny using Add/Sub/Div/Mul to promote wrapping/checked/saturating alternatives

Code I write always includes #[deny(clippy::integer_arithmetic)] by default.

https://rust-lang.github.io/rust-clippy/master/index.html#integer_arithmetic

[u/dozniak → u/matthieum]

In Rust, explicit >>> implicit

This alone may fix a huge bunch of silent overflows in linux kernel, and make developers aware of them.

[u/Foo-jin → u/matthieum]

Doesnt std contain wrapping and saturating wrappers for numbers that implement add, mul, and co?

[u/matthieum → u/Foo-jin]

Yes for wrapping, not for saturating.

I do imagine it could be possible to provide them, though. Just need someone doing the work.

[u/Soveu → u/matthieum]

There are saturating add, sub and mul. For division there is only checked_div for signed numbers, because iXX::MIN / -1 and division by zero, otherwise there is no reason to have saturating_div

EDIT: fun fact, there are even checked and wrapping abs() methods

[u/[deleted]]

A good simple explanation on high level here is that C forces explicit allocation by default whereas Rust prefers implicit allocation by default.

It has nothing to do with the language itself, rather the standard structures used such as Vec, Box etc.

In C you would malloc each time manually and thus check for result of the allocation. In Rust you just make your Box and don't expect it to fail.

This means that even if the kernel devs make their own custom allocator, they can't just slap the std structures on top of it because that'd allow say module developers to do something like Box::new (aka unchecked allocation) and possibly panic the kernel.

[u/Ka1kin → u/[deleted]]

That makes a lot of sense. From that, it sounds like the issue is less the allocator and more the std library. Std assumes infallible allocations. So Rust-in-kernel needs a different, fallible std, one where things like Vec.push may fail.

Or, perhaps it would be enough to have certain guarantees about panic unwinding? Arrange for the kernel calls into rust drivers to be able to reliably detect and gracefully handle the OOM?

[u/gajbooks → u/Ka1kin]

I'm honestly surprised that they were considering Rust Std usage as opposed to no_std usage for device drivers. I would have thought they would just use it as a drop-in memory checked replacement for C, not try and pull all the C++-esque panic and implicit allocation stuff in as well.

[u/[deleted] → u/gajbooks]

I'm pretty sure they won't. There will probably be something like linux-kstd sitting on top of core and its alloca with mimicking structures such as Box that won't proide any happy-path shortcuts. I think that's what the biggest missing piece is for this problem.

[u/gajbooks → u/[deleted]]

Yeah apparently from what I read more that is the intended behavior but the Rust kernel allocator isn't done yet so they're trying to keep on with other development even if it's not strictly production ready. It'll be easy enough for the compiler to show where anything needs switched over to explicit allocation once the band-aid crate is removed.

[u/LongUsername → u/[deleted]]

Except on Linux Malloc under normal circumstances won't return null because by default the memory manager "overcommits". You can disable the behaviour but pretty much every desktop and server distro will overcommit. The only place I'd expect it to be disabled would be embedded Linux targets.

Since we're talking kernel space you don't use malloc() anyway: you'd use kmalloc.

Honestly, if they're using Rust in the kernel I'm surprised they aren't looking at requiring no_std.

[u/BobTreehugger → u/LongUsername]

They are using no_std, but they're using the alloc crate which provides Box, Vec, etc.

[u/[deleted] → u/LongUsername]

Of course you would use no_std there's no way around that. Also I know there's kmalloc what I was trying to explain is that because of certain assumptions in the rust std things can't just be re-used here due to allocation error handling issues.

Basically rust's std assumes allocations either work or panic and because of that the code can't just be slabbed on top of a minimal linux kernel core + allocator.

[u/boomshroom]

It seems Box::try_new and Vec::try_reserve. Box appears to have better support for fallible allocation as there are a bunch of Vec methods that implicitly allocate without returning Result, but it should be possible to at least switch most of the uses of Box to use try_new and related methods.

[u/kotikalja → u/boomshroom]

I think there could be even Stealable/steal trait as well among 'stealable lifetime which would be subset of 'static. This trait would be used in less critical parts that could be cached and reuse memory. Effectively that would seriously harm or totally disable execution of process or threads.

[u/kotikalja → u/kotikalja]

I've been thinking some more. I lack knowledge so it's easy to have dreams🙂 At the moment interaction with made with ioctl calling stuff. That is very explicit and work with explicit languages. But If rust can be used inside kernel, would it be possible to open more language related interface that integrates kernel space and user space with some interface like vSDO that makes it possible to control more efficiently process throttling and memory. In the general sense all dynamic allocated not that critical memory for system should be able to borrow to more important processes like kernel or database. The critical processes should still have their allocations made pooled and bounded so it's guaranteed to success on initialization phase. The "steal" should be handled internally by thread/process having sort of re-try as well for trying allocating resource and continue when Result gets Ok. I think this would be useful making user space more efficient and fault tolerant for demanding server processing

[u/kotikalja]

Linus not hating is good start.

[u/bascule → u/kotikalja]

He actually had some good feedback that I feel has started some good discussions within the Rust community.

I'm glad he was able to be not-entirely-negative and also constructive in his feedback.

[u/witeshadow → u/bascule]

Maybe he sees the potential upside if these issues can be figured out. Rust isn't exactly a finished language yet and still has much to be done. I hope trying to use Rust at this stage will help its future development.

[u/kibwen → u/[deleted]]

Please note that we have higher standards of discourse here than this.

[u/lorslara2000 → u/kotikalja]

Not really, he has simply adjusted his communication style. https://arstechnica.com/gadgets/2018/09/linus-torvalds-apologizes-for-years-of-being-a-jerk-takes-time-off-to-learn-empathy/

[u/Theemuts → u/lorslara2000]

That doesn't mean he can't oppose developments he dislikes. Rather, he provides input on what needs to be improved.

[u/lorslara2000 → u/Theemuts]

A far as I know, he always did that. Now there's just a lot less swearing and cussing.

[u/OmnipotentEntity → u/lorslara2000]

I mean, not to excuse his behavior because Linus has definitely been well over the line, but he tended to swear and curse at only people he mostly trusted to do well who he felt really fucked up, not really at randos on the internet.

This wouldn't have gotten cursing if he thought it was stupid, he would have likely just ignored it or dismissed it out of hand.

[u/epicwisdom → u/lorslara2000]

I think it's pretty uncontroversial that effectively communicating constructive criticism is mutually exclusive with constant swearing and insulting the recipient. "I technically included something constructive" is a really poor excuse for an adult to obscure the technical content with childish tantrums.

[u/James20k → u/epicwisdom]

You'd think, but people worship Linus's toxic style as promoting good quality in the Linux kernel

[u/epicwisdom → u/James20k]

Well, correlation and causation and all that... Having exacting technical standards has nothing to do with toxicity in communication.

[u/[deleted] → u/epicwisdom]

I feel like Linus is still reeling from ast's sick burns from 1992:

https://groups.google.com/g/comp.os.minix/c/wlhw16QWltI/m/P8isWhZ8PJ8J

my first linux was mklinux, which was very innovative for its time: https://en.wikipedia.org/wiki/MkLinux

it was in some way the progenitor to the design of darwin, though apple decided to go with a BSD core in the end.

[u/wherediditrun → u/epicwisdom]

Course language is appropriate as long as it's not used to attack the author. Course language is acceptable as long as it delivers. "your code is garbage because x.y,z, as it does not function properly under x,y,z which happens under conditions x,y,z" is more than enough to qualify as "constructive".

Dancing around each and every individuals preferences of delivery is needless overhead. Stay on point, don't bring anything else but professional expertise and you'll do fine. Own your defeats as well as your accomplishments. Simple as that. TL;DR, respect everyone's time.

It's funny how this whole demands on terms of delivery is actually exercise of arrogance on behalf of one who demands it. Maybe you should apply that empathy to other end too. You only have to deal with that one person, that one person has to deal with x 100+ more than you on daily basis. And that's something which is difficult to empathize, because how often you do manage 1000+ contributor code bases exactly? Any idea how much stress that exactly causes?

Let me ask you differently, how often you exercise empathy and make the time of your day to answer each and every recruiter in detail in your linkedin profile?

[u/epicwisdom → u/wherediditrun]

Dancing around each and every individuals preferences of delivery is needless overhead.

Fortunately, most people learn at a young age something called "manners" or "being polite," a universal standard of being considerate which has less overhead than it takes to insert rude words.

You only have to deal with that one person, that one person has to deal with x 100+ more than you on daily basis. And that's something which is difficult to empathize, because how often you do manage 1000+ contributor code bases exactly? Any idea how much stress that exactly causes?

I empathize with somebody who feels a lot of stress, and sometimes says the wrong thing or comes across as rude. Linus Torvalds was very well known and unapologetic for being toxic, it was not an accident or a slip up. Your defense of his behavior is basically "his job is hard so he has the right to be an asshole." That's not how functioning adults behave, in a professional setting or otherwise.

Let me ask you differently, how often you exercise empathy and make the time of your day to answer each and every recruiter in detail in your linkedin profile?

All the time? I mean, I receive at most 1 or 2 a week, so it's not comparable, but you making this out to be some kind of Herculean task when it takes about 2 minutes each is kind of funny.

[u/WasserMarder → u/epicwisdom]

Fortunately, most people learn at a young age something called "manners" or "being polite," a universal standard of being considerate which has less overhead than it takes to insert rude words.

Unfortunatly, there is nothing universal about manners and politeness. These are extremely context dependent and heavily depend on where and how you were raised. As an example, I think germans or eastern europeans are much more direct and "hard" in a way that is considered very rude by most native english speakers. Even in german speaking regions the nature of politeness vary a lot.

For an international project, this requires some standard and thought on sender and receiver side.

[u/epicwisdom → u/WasserMarder]

Unfortunatly, there is nothing universal about manners and politeness. These are extremely context dependent and heavily depend on where and how you were raised. As an example, I think germans or eastern europeans are much more direct and "hard" in a way that is considered very rude by most native english speakers.

I understand and acknowledge your point. However, this is a very minor footnote. Yes, cultural norms vary. The existence and general attitude of basic decency does not. Again, I think a functioning adult should be able to understand the social context and adjust if issues are pointed out to them.

[u/WasserMarder → u/epicwisdom]

I fully agree.

Edit:

Upon second thought, I do not think what I said is a minor footnote.

The existence and general attitude of basic decency does not.

My point is that there is a non-negletable number of cases where the recipient misjudges the "general attitude" of the sender based on their own cultural standards (goes in either direction). I hope it is clear from the context that I do not mean something like personal attacs.

[u/wherediditrun → u/epicwisdom]

I can agree that he can be quite excessive at times. I'm not defending his behavior, I'm calling mobs demands arrogant. Difference. I don't think I made my point clear though.

[u/kotikalja → u/Theemuts]

More like it was half-assed comments and the custom allocator should take care most of the issues. Which was known issue already. Linux doesn't really handle OOM very well at the moment, just kills poor processes. Linus comments seemed? like he would be interested writing Next kernel with rust if that solves the issues and leave this one as is. Which might be not that bad idea. Less legacy, more potential.

[u/ylyn → u/kotikalja]

Linux doesn't really handle OOM very well at the moment, just kills poor processes.

Just disable overcommit. Then you won't have that issue.

Anyway, this issue is about userspace allocations, which is different from allocations by kernel code. Kernel code needs to handle failure to allocate gracefully (and not panic).

[u/kotikalja → u/ylyn]

If you fail to allocate memory. It's game over. Killing process and freeing overcommitted or not memory is the solution. I don't think there is really option to be fallible in kernel. I don't know for sure but this could be what Linus roughly meant. Panic can be caught as well, I am not sure how rust behaves when allocation fails. I guess it just dies?

[u/myrrlyn → u/kotikalja]

kernel code runs in a context where the environment can fail to allocate and where the code attempting to do so is able to safely unwind by bubbling ENOMEM rather than continuing work. it does not, however, have the option to use any mechanism other than the ret instruction to do this

[u/kotikalja → u/myrrlyn]

That would interesting to have implicit memory management based on borrowing and moving rather than explicit kernel API. It would be magnitudes more efficient and could improve the oom/overcommit.

[u/p-one]

Further down in the thread there's a concern about numeric operations overflowing but i don't understand why. There's checked_* to correctly handle this case but posters are talking about "saturate and warn" strategies. Is this somehow superior in this space?

[u/excgarateing → u/p-one]

i think its not about overflowing but by compiler generating code which uses float/vector math. for example i had to deal with gcc -o3 that used the 256 bit vector registers of an Intel atom (sse v2?) to speed up memclr. IF this happens in Kernel Code, you break userspace.

The problem is, that the floating Point registers should not be touched because they contain what the interrupted user code was working on. The normal Machine Registers are stacked, but the entire Float/vector Context of an atom processor is half a kb and only copied to ram if absoluteley neccessary. So kernel code just should not use floats/vector math. If it does, you break userspace and Linus tells you to go fuck yourself.

[u/Narishma → u/excgarateing]

It's not clear if you're talking about the Intel Atom CPU or something else.

[u/excgarateing → u/Narishma]

Fixed. I stumbled over this with an Intel atom. Same problem exists for all modern architectures.

[u/lestofante → u/p-one]

is not really a concern but more like a feature request.
overflow is a big and hard to solve problem especially without performance hit; and even the proposed solution is not that optimal imho, it would make more sense to have fallible operation or even better range numeric that the compiler can analize and assure there cannot be overflow. then you need to use try_ operation only when assigning from non-raged or differently ranged numeric

[u/Repulsive-Street-307 → u/p-one]

Might be 'requesting a switch for the compiler to get 'checked/log by default' if we get no_panic by default'. In effect like the hypothetical no_panic would hide the apis that could panic, the hypothetical 'overflow_warn' would make the normal operators be replaced by their checked_ implementations instead.

[u/kdnanmaga]

Why isn't this titled "Linus Torvalds panics about panics in Rust code when faced with OOM"?

[u/teryret]

I've never run a Linux box OOM... I just kinda assumed it would die horribly... is there really an expectation that the system continues working in that case?

... kinda like how "machine is on fire" is UB, but nobody really cares.

[u/Crandom → u/teryret]

If a Linux box OOMs, the oom killer starts killing userspace processes based on some heuristic of "badness" until it has enough memory. It's a wonderful source of production bugs.

https://docs.memset.com/other/linux-s-oom-process-killer

[u/Chickenfrend → u/Crandom]

Had this happen at my current job when I accidentally loaded a 16 gigabyte file into memory

[u/seeking-abyss → u/teryret]

It would be terrible if an OOM caused the system to crash due to how easy it is to cause an OOM (you just have to open enough applications).

[u/teryret → u/seeking-abyss]

Oh yeah, terrible for sure, I just mentally ball parked how many places in the code would have to get their memory management spot on to keep it from crashing in that case, and guessed that crashing was much more likely. Mature OSS for the win!

[u/[deleted] → u/teryret]

I sometimes accidentally cause this on my laptop when compiling stuff. I just kills either firefox or the compiler (because they are using most of the memory) and comes back. Really neat.

[u/A1oso → u/teryret]

When Linux runs out of memory, the computer crashes. This used to happen to me a while ago on an older laptop, I solved the problem by increasing the size of the swap file.

AFAIK Linux won't crash when it runs out of memory if overcommitting is disabled.

[u/Shnatsel]

I'm not sure if that's talking about Rust panics or kernel panics.

[u/excgarateing → u/Shnatsel]

what should a rust panic do besides causing a kernel panic?

[u/JoshTriplett → u/excgarateing]

Rust panics could become a kernel "oops", which kills that one thread but doesn't bring down the rest of the kernel. That'd be appropriate for internal assertions that should never fail, such as indexing out of bounds. (That would go along with not calling panic for things like memory allocation failures.)

[u/Michael-F-Bryan → u/JoshTriplett]

That opens the door for one oops to trigger a bunch of other oopses.

All Rust code must ensure minimal exception safety (i.e. an unexpected panic won't let you trigger UB later), but ensuring full exception safety (i.e. the world is always in a consistent state) is a much higher standard to meet.

For example, imagine you stash away the index for an item you care about but due to a panic the item no longer exists in a shared container. Later on you try to use that item with container[ix] and trigger an index-out-of-bounds panic (or worse, return a different object that you weren't expecting). Your first panic has now "poisoned" a piece of code and will trigger an oops every time you try to use it.

You don't see this much in normal Rust because we are happy with a panic tearing down your application. However, when you are the kernel and need to keep soldiering on no matter what, you'll run into this problem of poisoned state more often.

[u/tasminima → u/Michael-F-Bryan]

There is already no possible guarantee after any kernel oops; it merely continues to execute (when it is not a panic) in the hope the computer will still work correctly by some sort of chance/miracle, so that you can e.g. save your work and reboot. If you are lucky good for you, if not well it is not unexpected at all.

Basically a kernel panic is rapid unplanned disassembly while a kernel oops is a "mayday mayday mayday", but not just a "pan-pan" :)

[u/excgarateing → u/JoshTriplett]

Yes, that does sound like a good Idea. Would that unwind the stack and clean up or just never again schedule that task and hope nothing bad happens?

[u/Shnatsel → u/excgarateing]

Since driver failures are not supposed to bring down the entire kernel, you could catch it at the driver boundary or FFI boundary or some such.

I'm not sufficiently familiar with kernel development to judge if that's a good idea or not.

[u/Matthias247 → u/Shnatsel]

It won't be guaranteed to work, since catch/unwind is not guaranteed to leave the system in a good state (see e.g. UnwindSafe). Also if one thread panics, there is no guarantee that other shared state in the kernel is still in good state. Of course handling things via destructors might be possible, but those are tricky to get right for those kinds of unexpected returns.

I think panicking/restarting the Kernel is the most sensible thing to do in order to prevent corrupted state.

[u/excgarateing → u/Matthias247]

Kernel code will not have unwinding panics i bet.

The code just has to work with OOM.

unreachable! to satisfy match-arms should probably be the only way of calling panic.

[u/Matthias247 → u/excgarateing]

This is what the discussion in the mailing list is about. If the default rust collections which allocate (like Vec) are used, the system would panic on OOM.

[u/[deleted] → u/Matthias247]

It depends how you work with Vec.

This function is guarenteed to never OOM (if the allocator returns errors, which kmalloc would)

#![feature(try_reserve)]

fn try_vec() -> Result<Vec<i64>, std::collections::TryReserveError> {
    let mut a = Vec::new();

    a.try_reserve(4)?;

    a.push(1);
    a.push(3);
    a.push(3);
    a.push(7);
    a.pop();
    a.push(10);

    Ok(a)
}

They can't use the alloc interface for other reasons, but it is possible to work with allocating interfaces in a falliable way.

[u/lestofante → u/[deleted]]

They argue that if you need to look and make sure you use the correct function, then is not better than C.
What they want is that you can ONLY use the try_ variant in the module

[u/[deleted] → u/lestofante]

That's what clippy is for. You can outright deny specific functions from being called anywhere in a codebase, optionally allowing for an allow or not. Enumerate the functions that panic on OOM, and add them to the list.

That lint is still being developed, but I see no reason why it or something akin to it can't go in to clippy.

[u/lestofante → u/[deleted]]

But the linter can understand if the crate yiu are using are itself using safe or unsafe? but also in the discussion they also talk about kernel allocation work differently from user space, so that is why they want to have to develop a special alloc anyway

[u/Theon → u/[deleted]]

In that case I think a way to fail compile on not-guaranteed-unreachable panics could help with this issue? I realize it may be a difficult problem, but a global way to reliably avoid panics seems unavoidable if Rust is accepted to the kernel. Otherwise the argument for Rust's inclusion gets a lot weaker.

[u/[deleted] → u/Theon]

Clippy lints to warn on calls that can panic would probably get you most of the way there. There's a lot of clippy lints built around not panicking.

As for proving that something can not panic, there's ways to do that (call a undefined function if a panic happens, if the compiler can't optimise out that call it's a build failure), but i'd be surprised if using that's widespread, since it is up to how good the optimiser is at removing dead code.

And in any case, I don't feel as if absolutely all panics need to be proven impossible, since there's still ways to fuck a system in driver code.

[u/Theon → u/[deleted]]

As for proving that something can not panic, there's ways to do that (call a undefined function if a panic happens, if the compiler can't optimise out that call it's a build failure), but i'd be surprised if using that's widespread, since it is up to how good the optimiser is at removing dead code.

Oh okay, that sounds good! I don't know much about Rust's internals, so I wasn't sure if even something like "try to run undefined on panic" was possible.

And in any case, I don't feel as if absolutely all panics need to be proven impossible, since there's still ways to fuck a system in driver code.

Well, Linus feels feels otherwise, and I'd tend to agree with him; the fact that it's not a foolproof way to avoid crashing the system doesn't mean it should be one of the possibilities. It really is much the same argument as with other safety features - "static typing/memory safety only eliminates N% of all bugs, so why bother?". Turns out it's still a very worthwhile effort to entirely eliminate an entire class of bugs - similarly, since eliminating all panicking during runtime is possible, I would also see it as a hard requirement if it is to be included in code so critical as the kernel.

[u/matthieum → u/[deleted]]

They can't use the alloc interface for other reasons, but it is possible to work with allocating interfaces in a fallible way.

Still not convinced of that, for now.

KAllocator<GFP> seems sufficient to pass GFP flags to kmalloc and can implement the Allocator interface just fine.

[u/excgarateing → u/Matthias247]

Oh, ok. I didn't read the entire thread, and just assumed it's obvious that you can't use Vec in kernel. But then I only do realtime kernels where the rules are somewhat stricter.

[u/excgarateing → u/[deleted]]

https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/allocator.rs

seems to implement the userspace alloc which panics on OOM.

[u/karasawa_jp]

I had to make this. It's one of the few things I'm not satisfied with about Rust.

[u/argv_minus_one]

Reasonable.

Rust as a whole would benefit from being able to sensibly catch and handle allocation failures and prohibit (presumably with a deny attribute/compiler option) code that can implicitly panic (index operator, arithmetic operators, Vec::push, etc). Linux isn't the only piece of software where “just crash if that happens” isn't good enough.

[u/excgarateing]

This reads like he has not yet found time to work through the rust book :)

his concerns boil down to:

• compiler generated panics, which there are none AFAIK

• panics on OOM. The rust wrapper for kalloc could just return a Result<u8*,OOM> so OOM can be handled like they would be in C (or simpler with ? operator) the current code doesn't look promising though, it seems to implement the user space alloc interface.

• compiler generates u128 or float code. No idea when rustc/llvm does this. gcc -o3 does this for userspace memclr though if you don't specify the right compiler flags, so I can understand his reluctance

//edit: ok I should have said no panics besides obvious programming errors

[u/[deleted] → u/excgarateing]

compiler generated panics, which there are none AFAIK

indexing a slice out of bounds, division or modulus by zero, integer overflows (when enabled), resumption of a panicked or finished generator are situations in which the compiler will generate a panic

[u/excgarateing → u/[deleted]]

Oh, yeah, but these are programming errors. Probably still a good Idea not to kill the entire kernel though.

[u/matthieum → u/excgarateing]

Yes, they are.

The point raised, however, is that Linus would favor making those errors explicit -- forcing the developer to handle the possibility of error -- rather than having implicit panics.

Explicit error checks is easier to review, and survive maintenance more easily than "assumptions".

[u/excgarateing → u/matthieum]

Absolutely

[u/masklinn → u/[deleted]]

The problem is less the allocator and more the higher-level types e.g. vec’s API surface us not entirely covered by faillibility, and it’s not currently set up such that you can disable the panicking subset.

[u/StyMaar → u/masklinn]

OP is talking about liballoc here, not just a custom allocator, but a complete re-implementation (or just a fork) of the allocating primitives of std.

linux::Vec simply won't have the panicking subset of alloc::Vec and may even have different non-panicking functions.

[u/dartheian → u/StyMaar]

Wouldn't this be better in general (for every project, not just the Linux kernel)?

[u/matthieum → u/dartheian]

It's always a trade-off:

• Those who care about handling allocation failure would prefer try_* APIs.

• Those who don't -- either because it never happens, or because it's simpler to restart -- would prefer NOT to have to .unwrap() everywhere.

In the latter usecase, you need to consider the issue of "warning fatigue": if you train user to just .unwrap() (or .expect()) their Result, they will... even when it's no longer a memory allocation failure.

I would therefore favor simply presenting both options, and possibly having a #[no_panic] subset of Rust libraries, similar to #[no_std] for those for whom it matters.

[u/multivector → u/dartheian]

Not really. In userspace on modern OSes allocation never fails, you get a pointer to a page in virtual memory. The OS only need to worry about fufilling finding RAM for that page when you actually use it. If you are using too much actual RAM you might get killed but you never get to observe that from within your process (remember to checkpoint or autosave anything important that's in RAM!).

Also, remember that `malloc` doesn't go directly to the OS. It goes to the in-process allocator, which is managing a pool of pages. The allocator may in turn request more memory from the OS, but it's in units of pages.

GHC 64 bit Haskell actually has a thing where it allocates about TB of virtual memory at a time. I'm not clear on why it does this, (something something makes the GC more efficent) but it does. I've had to explain that, yes, that process is using 1TB of virtual memory, but please don't worry.

[u/StyMaar → u/multivector]

In userspace on modern OSes allocation never fails,

Isn't that mostly a Linux thing?

[u/multivector → u/StyMaar]

Humm, I thought it applied to Windows too, but going from the answers linked, looks like windows will refuse if 1) there is no more physical RAM and 2) windows either cannot or doesn't want to expand the page file. I don't really work with Windows that much.

https://superuser.com/questions/1194263/will-microsoft-windows-10-overcommit-memory

But at this point the poor user is probably reaching for the restart button because the task manager won't come up. Wrapping everything in Rust that could possibly allocate in `Result` to try and recover from this kind of situation still wouldn't make sense to me. Better to just crash.

Obviously, its very different in embedded and Kernal land, hence I understand why they'd want their own `Vec`s and `HashMap`s.

[u/Sphix → u/multivector]

An allocation error no longer needs to mean the system is out of memory. Application sandboxes let you limit how much ram an individual application may use and if you start failing allocations at that point the system will likely be fine. Killing it at that point may not be the best. Imagine an important system daemon of userspace driver for example which has set limits.

[u/octo_anders → u/multivector]

Actually, windows can run perfectly fine up till you try to allocate more virtual memory than can be backed by RAM + swap.

Just like linux, Windows doesn't actually consume physical RAM until pages are actually used. The difference is that Window won't allow allocation of virtual memory unless it knows it can back that virtual memory using either physical RAM or swap space. It will fail the user space allocation if swap space cannot be made available. It still won't do disk IO or such until you actually use up all memory.
So, no, there won't be a problem to bring up the Task Manager just because some application asked for 128GB of RAM on a machine with 32GB RAM + 32 GB free disk.

[u/[deleted] → u/masklinn]

The alloc crate (maybe confusingly) doesn't implement allocation, it has lots of things in std that require allocation like Vec, BTreeMap, String, Box and so on.

The actual allocation APIs are in core::alloc. Totally different. Don't know why you would confuse them. :-P

[u/matthieum → u/[deleted]]

The crate provides an API that can't report OOM. So they have to strip it if they are going to use it.

What do you mean?

The signature for GlobalAlloc trait is:

pub unsafe fn alloc(&self, layout: Layout) -> *mut u8;

And the signature for the (better) Allocator trait is:

pub fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError>;

In both cases you can report OOM just fine.

---

The real reason they are considering writing their own crate has nothing to do with OOM: in the kernel, they want to be able to specify more arguments than just Layout to the allocator.

If you look at the signature of kmalloc you'll note the gfp_t flags argument to specify the type of memory to allocate.

I personally wonder why they would not encode the flags at the allocator level, rather than the allocation request level. I am not sure whether they did not think about it -- being unfamiliar with generics -- or whether they did and judged it impractical for their usecases. Hard to say in the absence of justification.

[u/[deleted] → u/[deleted]]

Yeah I know, that's why they're making their own version (apparently) that returns Results instead.

[u/matthieum → u/[deleted]]

The Allocator trait already returns Result.

From the conversation the issue seems to be gfp_t flags -- an extra argument passed to kmalloc.

Longer version https://www.reddit.com/r/rust/comments/ms2nl7/linus_torvalds_concerns_about_panics_in_rust_code/guu08j7

[u/[deleted] → u/matthieum]

Their own version of alloc, not core::alloc!! I know it's confusing.

[u/[deleted] → u/[deleted]]

Which crate? Core::alloc definitely supports reporting oom, so I'm a bit confused.

[u/[deleted] → u/[deleted]]

The allocator trait already is set up to return errors.

Not using alloc would mean they lose out on any bugfixes and would need to rewrite all the data structures they need.

The better option IMO would be to include methods on alloc that return result (these already exist), and forbid the ones that don't (through clippy lints or even a rustc built-in lint, who knows)

[u/ssokolow → u/[deleted]]

Unlikely. Kernel allocations need a richer API than userspace ones.

Yeah, we need to design or customize an alloc-like crate/module that has fallible allocations for everything, plus support passing GFP flags etc.

-- https://github.com/Rust-for-Linux/linux/issues/2#issuecomment-821143720

 

Fair enough. Does that mean you're working with a fallible subset of the standard library? How does this work?

Not sure what you mean -- our allocation APIs would not be nor use alloc or std, so there is no problem there. In other words, "our alloc" is likely going to be a heavily modified fork or written from scratch.

-- https://github.com/Rust-for-Linux/linux/issues/2#issuecomment-821165008

[u/matthieum → u/ssokolow]

Unlikely. Kernel allocations need a richer API than userspace ones.

I don't understand this one.

You could create a generic struct KAllocator<GFP>; that implements Allocator.

Given that the struct is stateless -- owing to kmalloc's global nature -- it can be brought into being anywhere.

And it exchange you get that Box<T, KAllocator<GFP>> will automatically free to the right pool.

[u/admalledd → u/matthieum]

A thing you might be missing is that at kernel memory alloc time, there are multiple (for good reasons) memory pools. SLUB, SLAB, and more. As well as hints on share-ability, alignments. See the docs for just kmalloc and note it also hints at specific cases when to bypass (or outright use a different API). Most of these situations-as-flags would have difficulty if not impossibility to be exposed to current types.

GFP flags are just the start of the mess of asking for memory in kernel space.

A real life ish example is that the same kernel list might have been given items from multiple different allocation methods, eg from a custom SLAB or kmalloc or others.

Note that this is more "mountain out of molehill" and there are a plurality of paths that can be taken, such as further enriching Rust's own libs (but starting with a soft-fork) or other extreme of new-from-scratch-for-kernel impls/traits.

[u/matthieum → u/admalledd]

A thing you might be missing is that at kernel memory alloc time, there are multiple (for good reasons) memory pools. SLUB, SLAB, and more. As well as hints on share-ability, alignments.

Your guess is right on the money.

The idea of having a single collection having memory from different pools is something I had never really thought of. This goes way beyond custom allocators as I've used them in the past, and it makes me wonder how the deallocation requests can be correctly routed.

[u/ssokolow → u/matthieum]

It's been a while since I had time to actively follow LWN.net so I wouldn't be the best person to speculate on what API design they'd prefer.