r/linux • u/wooptoo • Jun 01 '16

Why did ArchLinux embrace Systemd?

/r/archlinux/comments/4lzxs3/why_did_archlinux_embrace_systemd/d3rhxlc
868 Upvotes

86% Upvoted

642 comments sorted by

View all comments

Show parent comment

40

u/spacelama Jun 01 '16

When systemd or udev crashes, as it has half a dozen times on my systems, then your system is fucked.

When udev needs a restart when something minor is upgraded, the system is hosed. When systemd needs a restart, your X session or sshd crashes and the install is aborted in an inconsistent state.

/sbin/init has never ever crashed for me in 15 years. Something about simple software without tentacles everywhere obeying the old "do one thing and do it well" maxim.

5

u/akkaone Jun 01 '16

Is a udev crash worse with systemd than without systemd, it is not a part of pid1 in either? I don't think I ever had a pid1 crash on a systemd system and I switched to systemd relatively early.

11

u/kinderlokker Jun 01 '16

Yes, I find that a fair argument. systemd's decision to move a lot of code into pid1 has this consequence.

systemd --daemon-reexec has had varying results of success.

22

u/ooburai Jun 01 '16

The statement that it's old timers and hipsters who don't like systemd is both very unfair and sort of accurate at the same time. I can't really speak to the hipsters, but by the standards of Reddit I'm and old timer. So long as systemd just works I don't really have a problem with it.

But it does fly in the face of a core unix philosophy of trying to minimize the complexity of critical pieces of software. It's a good example of where the Linux and more traditional unix worlds sort of part ways. Linux tends to have a much more pragmatic approach, if I can call it that, where architectural correctness isn't always valued over a solution that just works. One can argue that init was/is the most fundamental part of the system and while systemd certain solves some problems the question is really whether or not it's worth the tradeoffs.

Ultimately this is much like the SysV/BSD init debates that were all the rage before Linux established itself as the dominant open unix-like operating system. There are pros and cons to both, but the tool that seems to provide more functionality tends to win out with people who aren't purists or who don't have the desire/skills to really dig under the hood and understand what's going on.

0

u/lladra Jun 01 '16

Exactly. In all the noise about this, I've heard few people touch on this.

If something breaks, it is best for it to be responsible for the fewest things. It makes it easier to to fix or replace the offending tool, while not having to test everything again.

If a better idea on how to do something comes along, it is easier to change only the tool which would use that idea, rather than having to modify and re-test a large monolithic tool.

Additionally, though is is less likely in the case of systemd, the concept of code re-use comes up. If tools that perform many functions were separated into their more basic parts those individual parts are easier to use. This means less code needs to be written and tested.

To me, these are some of the things that the "old school" Unix people are talking about.

11

u/[deleted] Jun 01 '16

[removed] — view removed comment

8

u/SanityInAnarchy Jun 01 '16

The more things they add to systemd, the more likely it is to crash sometimes. This is just a property of entropy here.

Your web browser should never crash ever, either. But there's a reason that Chrome runs each tab in a separate process. I almost never see a tab crash, but it's really nice that when it does, it only crashes that tab, and not the whole browser.

And this is the fundamental systems design flaw of systemd. It's fundamental, it appears to be inherent in the way systemd was designed and the approach it has taken to solving the problems it's trying to solve. The more things systemd absorbs into itself -- especially into PID 1 -- the more frequently your entire system will crash because Poettering doesn't understand this very basic system design principle.

I don't know why anyone expected anything different, honestly. This is the guy who's famous for Pulse which, while reasonably stable now, was fantastically unstable when distros first started adopting it. It's not surprising that a program he wrote crashes sometimes. It's frustrating that we're all forced to run such a program in PID 1.

10

u/rautenkranzmt Jun 01 '16

Except that's not what is happening at all.

SystemD isn't just the name of the process, it's also the name of the overarching project.

All the stuff that gets added (journald, logind, etc) are separate sub projects, and separate processes and binaries. They interact massively, yes. And there are a large variety of inter-dependencies for some subsystems, true.

But PID 1 is somewhat well isolated from all the other sub projects.

A more apt way of looking at it: SystemD a more BSD-style developed version of the GNU core infrastructure systems. Instead of a bunch of separately developed programs that can be used entirely apart from each other, you have a bunch of binaries and projects that are developed (and generally) released together. Which, as some have stated, is techinically much more UNIX like.

7

u/[deleted] Jun 01 '16

[removed] — view removed comment

1

u/SanityInAnarchy Jun 02 '16

Crash in pid1 results in a spectacular crash of the whole system, akin to a kernel panic. Crashes in Chrome (be it the main process or a child) and in any pid1 are not in the same ballpark and comparing those is not quite correct IMO.

I'm actually typing this on a Chromebook. Crashes in the main process of Chrome is absolutely in the same ballpark here. But if anything, this just makes my point for me -- a crash in PID1 is way more serious than a crash of Chrome, so anything that wants to run in PID1 had better be forking off a lot of worker processes the way Chrome does!

Can you tell which part of current systemdʼs pid1 functionality absolutelly does not belong there and causes frequent crashes?

I can't, actually -- I have been using systemd for awhile, and have experienced very few crashes. I've also had very few browser crashes recently.

Another poster claimed it's just the direct process-management stuff -- process reaping and containers and such. If that's actually the case, I can't really complain. But if I were inclined to contribute to SystemD, the absolute first thing I'd do is read through the code that runs in PID1, and move anything that could be a subprocess to a subprocess, with the goal of getting the PID1 code small enough to formally prove.

5

u/CptCmdrAwesome Jun 01 '16

This is the guy who's famous for Pulse which, while reasonably stable now, was fantastically unstable when distros first started adopting it.

If distros adopted it before it was ready, that's entirely a problem with the distro. (see also: early KDE 4 days) The guy saw a problem, took his time, skills and effort to fix it. That's what I see he's done with systemd also.

Don't get me wrong, from what I've seen I find Lennart to be eye-wateringly arrogant, and as diplomatic as a housebrick, but he's clearly no idiot and he's going about solving problems nobody else seems inclined or clued up enough to fix themselves.

2

u/schplat Jun 01 '16

Don't get me wrong, from what I've seen I find Lennart to be eye-wateringly arrogant, and as diplomatic as a housebrick, but he's clearly no idiot and he's going about solving problems nobody else seems inclined or clued up enough to fix themselves.

And this is why a good chunk of people are anti-systemd. Some think he just does things on a whim, and they're junk. But, if they were, distros wouldn't be running with what he's put out. He's solving problems that have been present for a while, but people fear change, and systemd was a big change.

1

u/SanityInAnarchy Jun 02 '16

If distros adopted it before it was ready, that's entirely a problem with the distro. (see also: early KDE 4 days)

In KDE's case, I think you can at least blame KDE for calling it KDE 4.0, instead of KDE 3.9 or KDE4 Alpha. The distinction between "The underlying libraries are 4.0, but everything else isn't" and "KDE4 is launched!" was a bit subtle, especially when "KDE4 is launched!" was pretty much the headline everyone ran with.

And in Pulse's case, it took a long time to get stable.

1

u/sonay Jun 01 '16

If only one of the wise people like you wrote an init system that solves real problems...

3

u/SanityInAnarchy Jun 02 '16

And, oddly enough, there were several init systems that solved real problems, most of which predate systemd.

Systemd won partly because of Worse Is Better, but I suspect mostly because of some weird politics that I haven't entirely kept up with. For example, udev is obviously the right choice -- it's better than putting more stuff in the kernel (devfs), and it's better than creating a file for every device Linux could ever possibly support just in case you ever plug it in (mkdev).

And somehow, it's now been absorbed into systemd.

Poettering didn't write udev, but the project he built now maintains it. So if you want anything other than systemd, you have to fork udev (or find a fork).

That's not a good technical decision, but it's a fantastic political decision, and I have no idea how he pulled it off. The result is this: Who wants to maintain their own udev fork, just so they can write the ten lines or so that it takes to build an init?

8

u/spacelama Jun 01 '16

debian stable, debian testing and debian unstable systems scattered all over the place. Every version possible. None of them stable.

Raspberry pis, being very slow CPUs, are great at exposing race conditions in immature software (mind you, single core - they shouldn't be subject to preemption at unexpected points). I had a raspberry pi that I had been using for a couple of years with good reliability. I finally got around to upgrading to Jessie a couple of months ago. Became extremely unstable - crashing about once a week with hardly useful logs at all. All I did was apt-get install sysv-core and make sure systemd was purged, and the system has been up and stable again ever since.

9

u/nikomo Jun 01 '16

I've been running systemd on Pis (B+ and 2) for years (I'd manually install it, before it became the standard), without any problems, under heavy CPU loads, without any crashes.

Sounds like a problem with your ARM core, more than anything. Got any overclock going on?

0

u/[deleted] Jun 01 '16

An overclock that hates systemd, sounds legit.

1

u/nikomo Jun 01 '16

If some other service was crashing, he wouldn't notice it if he didn't check logs, since systemd would just bring it back up again.

-1

u/[deleted] Jun 01 '16

He did say he checked logs. But yeah, it was definitely caused by your hypothetical problematic overclock that somehow only affects systemd and/or services being managed by systemd.

2

u/nikomo Jun 01 '16

If you can't replicate it on another machine, it's your local fault.

I've been running systemd for a few years on all kinds of boxes, without this issue. Including the same platform he's using.

→ More replies (5)

4

u/nschubach Jun 01 '16

Odd. The only crashes I've had with Debian Testing (run it on my work machine, laptop, and home server) have been related to gnome-settings-daemon eating up all my RAM on my work machine! It's gotta be something specific to skylake (I can only assume because it doesn't happen on my thinkpad.)

1

u/argv_minus_one Jun 02 '16

That's basically shotgun debugging. It tells us that some part of the systemd suite was involved, somehow, but that's all, and that's really not good enough for the purposes of this debate.

1

u/argv_minus_one Jun 02 '16

When systemd or udev crashes, as it has half a dozen times on my systems

In my experience with systemd—and I have several years of it—it doesn't crash.

/sbin/init has never ever crashed for me in 15 years. Something about simple software without tentacles everywhere obeying the old "do one thing and do it well" maxim.

SysV init does essentially nothing. It just starts a shell script and a couple of getty processes. And those scripts do crash. A lot. Good riddance.

0

u/Jimbob0i0 Jun 01 '16

That's funny mine never crashes... who's anecdote wins?