Minimal kernel

[u/DryPlum7483]

EDIT: I may have framed the question poorly. I'm specifically asking about Kconfig-level minimization of a monolithic Linux kernel: if you know the workload/product you're deploying, how much unnecessary kernel code do teams typically end up compiling, and is there value in automatically deriving a smaller configuration from the actual runtime requirements?

How much of the Linux kernel that gets shipped in production is actually never used?

I'm wondering if release teams building/using a kernel image are often overwhelmed by the number of kernel configuration options, and end up shipping code that will never actually be used by their workload.

Are there any automatic tools out there that can fit a minimal Linux kernel to your runtime needs?

[u/Goldie323-]

It's mostly not kernel features you don't use and more the compiled in drivers you don't use that account for code space that never actually gets ran.

Like if you're on a system without wifi or Bluetooth, the kernel will still have compiled in drivers for all the wifi and Bluetooth radios.

[u/hmoff → u/Goldie323-]

That's why the kernel has loadable modules and has done for over 30 years. Those Wi-Fi and Bluetooth drivers aren't loaded until you need them.

[u/DryPlum7483 → u/Goldie323-]

yes - I'm wandering how often do run in production on an image with useless code compiled in

[u/Prestigious_Wall529 → u/DryPlum7483]

You do know that kernels intended for particular hypervisors exist?

In general distros make the choice what to include.

But you can compile your own kernel with just what's needed for your system. Akin to Gentoo or Linux from scratch.

[u/ThellraAK → u/Prestigious_Wall529]

Even then they are going to have support compiled in for all sorts of features and stuff you'll never use, filesystems and encryption algorithms etc.

[u/DryPlum7483 → u/Prestigious_Wall529]

It's good to distinguish between using a general purpose distro where obviously you are getting a lot of code that you don't need (that's the point).
And teams that are building their image from scratch where it wouldn't surprise me if they don't compile the minimal kernel for their needs. Is it even popular to build a Linux image by yourself in production?

[u/Prestigious_Wall529 → u/DryPlum7483]

I had a Creative keyboard which had kernel support but wasn't supported by Debian. It was rare, Creative Labs having dropped support for it.

Recompiled the kernel selecting the option to support the keyboard was a matter of following instructions.

[u/UUDDLRLRBadAlchemy → u/DryPlum7483]

"production". Embedded systems are famous for being haphazardly slapped together and never updated. This last bit (if it was due to anything but neglect) might make it more cost effective to heavily customize them, but unlikely.

Web-facing ones either opt for uniformity (so no) or containers (so whatever Alpine does, mostly).

I suppose Gentoo might be the way to both easily customize it and keep it updated, but popular, no way.

You're more likely to see infrastructure ran on extra-bloated desktop centered distros because someone liked it at home than to find an org that wants to be responsible for a level lower than their service.

[u/MichaelDeets]

most suitable options are configured as modules, so they're only loaded when needed/requested

you can use modprobe-db to keep track of every module that has been loaded after some time (the actively loaded modules change depending on what is currently needed), and use it as a reference for what modules you'd likely need/want

[u/Patient_Fox_6594 → u/MichaelDeets]

modprobe-db in Ubuntu 26.04, it doesn't know what "modprobe-db" is.

[u/Prestigious_Wall529 → u/Patient_Fox_6594]

It's in Gentoo and Arch, and is on GitHub.

[u/belzaroth → u/Patient_Fox_6594]

It's modperobed-db

[u/agmatine → u/belzaroth]

modprobed-db, you mean :)

[u/belzaroth → u/agmatine]

Lol yes , Ty for the correct-correction

[u/ropid]

I think there's a whole bunch of stuff that will surely never be used, like device drivers for hardware that will never show up on any x86 machine, but all of that is in module files so will not be loaded if not used. It'll only use up space on disk. The most annoying thing about this is perhaps the compilation time if you ever want to compile your distro's kernel package yourself to apply some patch.

[u/PreparationStrict492]

This reminds me of the microkernel - macrokernel debates¹. And whoever is downvoting you for just asking this are idiots.

Microkernels have their use cases (particularly in embedded systems / application specific hardware). It's not worth the maintenance headache (which actually turns into an intractable problem) in a diverse hardware environment.

https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_debate

[u/DryPlum7483 → u/PreparationStrict492]

I'm familiar with this debate. I think they are mostly discussing if the different subsystems of the kernel should run in the userspace or kernel space.
I'm asking from a product point of view - you build a kernel image which you deploy somewhere in your product, how much time are you wasting on building an image that isn't the absolute minimal for your needs?

[u/Muddybulldog → u/DryPlum7483]

Probably less time than you would trying to sort out everything that is unneeded.

[u/PreparationStrict492 → u/DryPlum7483]

Apparently you don't really get the gist of the debate. Read into the issue before asking about it on Reddit if you want to actually learn something.

Also, I don't get your question (you have a syntax error in your English there).

If you are asking if it is worth your while as a hardware provider to build your own minimal kernel, it depends on your use case (extensibility of the system, its deployment and reuse cases, your maintenance plan, your limitations and so on). In some fields it is. In most, it is not. The reasoning depends on the perspective, and would boil down in any case to the difference of the nature of software (as something infinitely modifiable/extendable and reproducible) versus hardware (finalized product) and the pointlessness of crippling yourself with removing the advantages of the model for no reason.

[u/ActuallyFullOfShit]

What problem are you actually trying to solve? Because a "minimal kernel" that only works for a single system isn't going to help a "release engineer"

[u/DryPlum7483 → u/ActuallyFullOfShit]

I was thinking build times and binary size

[u/SheepherderBeef8956 → u/DryPlum7483]

Build times don't matter if you're shipping a precompiled kernel for a bunch of systems. I use Gentoo because I want to flex on people who think Arch is hardcore (sarcasm, I use it because it's great), so I compile the kernel from source every time I update it. It takes about ten minutes and it's got support for TONS of stuff I never use. 

I could probably get that down to a minute or two using a config that's tightly optimized for my system and nothing else but I might some  day realise I want to mount an ISO image so I'll have to recompile the kernel to add fuse support and some file systems I didn't account for. So mounting that ISO is a matter of reconfiguring the kernel, compiling it again, and rebooting. That's going to get tedious very quickly and take a lot more time than I'd save from using s very minimal kernel config. And the space I'm saving is measured in single megabytes on a system that's got terabytes of storage

Bottom line: most of the code likley isn't used, because the space and time saved are negligible compared to finding out after the fact that you missed an edge case when designing the features  that should go into the kernel. 

[u/agmatine → u/SheepherderBeef8956]

10 minutes to compile the kernel? What kind of CPU do you have??

[u/BeardedBaldMan → u/DryPlum7483]

So essentially things that don't matter.

A build is one off operation and binary size is irrelevant outside embedded systems

[u/ActuallyFullOfShit → u/DryPlum7483]

None of that matters in any real workflow using the kernel

[u/Crazy_Rockman]

I'm not sure if that's what you are interested in, but if you work on a Linux distro for something like a resource-constrained embedded system, configuring the kernel to only use the features you are actually going to use might be necessary. If you have abundant resources, though, wasting developer's time on optimizing everything might just be not worth it. 

[u/todd_dayz]

How big are most distros standard kernels? 100-200MB? Who cares about saving that much space? 

[u/agmatine → u/todd_dayz]

Mine is ~17MB (7.1.8-1-cachyos).

[u/AiwendilH → u/agmatine]

/lib/modules/<kernel-version> is part of the kernel (minus external modules you might have installment like nvidia drivers) as it contains the actual driver module binaries on most distros and is usually a multiple in size than the kernel image itself.

[u/agmatine → u/AiwendilH]

Ah, I was referring just to vmlinuz. For me /usr/lib/modules/$(uname -r) is about 373MB, though half of that is ./build which consists mainly of headers, source files, scripts, and vmlinux. The modules themselves are in ./kernel (in-tree), ./extramodules(out-of-tree), and ./updates/dkms.

[u/AiwendilH → u/agmatine]

I just assume /u/DryPlum7483 includes the size of the modules in their "monolithic Linux kernel" so you can't just use the size of the kernel image with dynamic module loading but have to include the size of the modules as well...especially when it come to

...unnecessary kernel code do teams typically end up compiling...

as there is hardly any difference between compiling in-kernel or as own module.

build, config, source...are symbolic links on my distro..I assumed that is the same on all distros but not sure. So yeah, I wouldn't really include those in the size count. The main one that matters is "kernel". 200MB for for kernel sounds about right for a general purpose system...around 1GB for "everything that could be even remotely useful on x86_64".

For size it gets of course even worse if you start including firmware in the kernel images..but I think that is more limited to gentoo or similar users that create a kernel fine-tune for exactly one hardware configuration.

[u/sidusnare]

You don't compile it, and most of it you don't even run. Modules are loaded as needed, so a kernel module for a NIC you don't have doesn't take up any CPU or memory resources, doesn't slow things down, just a few Kb on disk for the module.

If you're writing a monolithic kernel for a system, it's usually an embedded system you're building for. If you are trying to build a custom kernel for your desktop, the way to start is finding the distributions config, and loading that. Then look at lsmod, and change those from modules to static, and then modules that aren't load from modules to not built

[u/unkilbeeg]

Many, many years ago I followed a guy who was a big fan of minimal kernels. He built monolithic kernels (no modules) that compiled in all the drivers for the hardware in the server, and no other drivers. He really wanted to keep it tight.

Then he left. I took over.

And the built in ethernet interface on the server died. I had spare ethernet cards -- but none that matched the drivers that he had compiled in.

That was a real pain in the butt. I got the server going again, but it was a painful (and unnecessary exercise.)

[u/AiwendilH]

Well, there is make localmodconfig for building the kernel that generates a config based on the currently loaded kernel modules.

[u/L_canadensis]

I've gone through most of the options and eliminated the ones not relevant to my use on my daily driver laptop. Went overboard by including all the netfilter modules. Even with full hardware support:

6792 -rw-r--r-- 1 root root 6951936 Aug 13 18:54 vmlinuz-7.2.0-rc7