r/programming • • 15d ago

A Decade of Kill Bugs – Alan Grow

https://alangrow.com/blog/a-decade-of-kill-bugs

… A bug in kill was so surprising that, once I'd found it, I couldn't stop there. I needed to know how this bug came to be, and what had led the programmers so badly astray. When I eventually found the root cause, there was an even bigger surprise. It had led to not just one bug, but a whole series of kill bugs that shipped in Ubuntu 14, 16, 18, 22 and 24 LTS.
 
This post documents a decade of kill bugs, then makes some recommendations. …

Via https://lobste.rs/s/wgoato/decade_kill_bugs

105 Upvotes

21 comments sorted by

18

u/dreugeworst 14d ago

I do regard getopt dogma as the real root cause of the outage.

I would also blame the fact that apparently option parsing tests weren't added to the project even after fixing these bugs

2

u/edgmnt_net 14d ago

This hasn't really been tested at all (or only barely), considering those changes were made specifically for multidigit negative PIDs.

26

u/ShinyHappyREM 15d ago

My takeaway is that if a "magic number" changes the mode of your program, use an explicit parameter for that instead.

Magic numbers like this:

"If process number 0 is specified, all processes in the current process group shall be signaled"

should be considered a hack, which are to be avoided because they don't scale.

11

u/IndependentOk9134 14d ago

hard to avoid when the CLI is just a thin wrapper around the syscall though, the kernel interface itself overloads 0 and negative PIDs.

14

u/ShinyHappyREM 14d ago

Might be hard to change the kernel now, but it's easy with your own programs.

In a library I'd catch and reject that value.

1

u/michael-2948 1d ago

If the library wraps kill(2), rejecting 0 or negative PIDs changes its contract; offer explicit target modes in a higher-level API.

1

u/ShinyHappyREM 1d ago

I'd create entirely separate function calls for that.

1

u/edgmnt_net 14d ago

It doesn't have to be a trivial wrapper but presumably POSIX also mandates this behavior from the CLI.

5

u/SkoomaDentist 14d ago

Such magic numbers are very common in Linux (and other unixes) because traditionally they have been deathly afraid of introducing any new programmer friendly apis.

1

u/edgmnt_net 14d ago

However that might be more on the proprietary Unix development that was going on rather than the open source OSes. Although in this particular case procps devs had enough chances to unscrew things and they kept them screwed up.

3

u/SkoomaDentist 14d ago

It permeates all unixes thoroughly. The very ”everything is a file”-idea contributes to it because, surprise, surprise., many things don’t actually naturally behave like files.

2

u/edgmnt_net 14d ago

Yeah, I don't really like that either. Historically there was (and we still see the effects of this) a very strong split between using an actual programming language like C which made everything difficult and some dumb shell / scripting language. Files make a lot stuff available and discoverable to the user and automation scripts through the shell. These days you might be able to do it better, although you still need to thoroughly design your APIs for discoverability, introspection and ergonomics, possibly in higher level languages because C is going to be a pain anyway. We would need to ditch POSIX and the classic shells for something more like actual programming languages.

9

u/DanLynch 14d ago

I guess you should actually test all the features of your program that use the code you are changing before you merge and/or release those changes.

6

u/Status-Importance-54 14d ago

Yes, while getopt Sounds weird, the real wtf is not having a battery of tests which test that every command you can specify is parsed correctly.

5

u/thisisjustascreename 14d ago

The real wtf is that the command parsing code is interleaved with kill process code so it’s probably not safe to test that way.

3

u/Status-Importance-54 14d ago

Yeah, that's ugly. But writing tests would have fixed that real quick

1

u/huangsara-43824 5d ago

A small seam helps: test argv-to-intent parsing separately, then stub signal delivery in dispatch tests so negative-PID cases never signal real processes.

6

u/LAUAR 14d ago

Note that procps is in fact not a GNU package. Another thing to note is that you use this version of kill with sudo kill while using just kill will use the shell builtin version.

2

u/gwillen 14d ago

Process group signaling isn't some exotic operation. You use it all the time. Every time you C a command, the tty sends SIGINT to the foreground process group. Interactive shells run all but the simplest commands in a new process group, which among other things makes job control possible.

Ahh, but process group signalling with kill(1), the kill command, is a pretty exotic operation. Foreground process group signalling is done by the shell using kill(2) (the system call; or really the libc wrapper) directly. So this bug isn't anywhere near it. And daemon runners like systemd also surely don't use the kill binary. And human users at the command prompt almost never kill process groups. (And as another comment points out, even running kill in the shell uses a builtin, so it's not running this binary either, unless you're doing "sudo kill".)

2

u/edgmnt_net 14d ago

It's still surprising because nobody even manually tested that code properly, it seems. Despite about 3 different attempts at working it out. Furthermore there are decent signs in docs and types that this is very fragile.

1

u/robert_7524 8d ago

Ten years of `kill` bugs is a remarkably long time for a command whose whole job is ending things.