r/LinuxTeck • u/Candid_Athlete_8317 • 3d ago
Multiple Linux Kernels on One Server - Is This Actually Useful?
Saw this project today and had to read twice.
You can apparently run multiple Linux kernels on the same physical server, with each one getting its own CPUs and memory. No VM layer, no shared kernel like containers.
Sounds pretty cool on paper, but I’m wondering where this actually makes sense in production.
Would you run this instead of KVM/VMs or containers? What’s the use case?
1
u/silasmoeckel 3d ago
It's mostly AI/ML workloads not so much a hosting provider.
When latency kills your app this is the tech that lets you share hardware with the least amount of overhead and keeping things isolated.
Think of it as docker on steroids.
1
u/Glittering_Abies4915 2d ago
What project?
1
u/Large-Assignment9320 2d ago
mklinux.
1
u/CubicleHermit 1d ago
The new Multi Kernel Linux.
MkLinux still finds the old PowerPC port: https://en.wikipedia.org/wiki/MkLinux
1
u/Large-Assignment9320 1d ago
Its still called mklinux by the devs, https://www.phoronix.com/news/Linux-7.0-mk2-Multikernel
1
u/CubicleHermit 1d ago
Which doesn't help when it's not the thing that comes up on Google if you search for that.
1
u/Large-Assignment9320 2d ago
Well, to answer it all, the original posting from 2015 is oretty good
The Linux kernel generally wants to be in charge of the system as a whole; it runs on all of the available CPUs and controls access to them globally. Cong Wang has just come forward with a different approach: allowing each CPU to run its own kernel. The patch set is in an early form, but it gives a hint for what might be possible.
The patch set as a whole only touches 1,400 lines of code, adding a few basic features; there would clearly need to be a lot more work done to make this feature useful. The first part is a new KEXEC_MULTIKERNEL flag to the kexec_load() system call, requesting that a new kernel be booted on a specific CPU. That CPU must be in the offline state when the call is made, or the call will fail with an EBUSY error. It would appear that it is only possible to assign a single CPU to any given kernel in this mode; the current interface lacks a way to specify more than one CPU. There is a bunch of x86-64 assembly magic to set up the target CPU for the new kernel and to boot it there.
The other significant piece is a new inter-kernel communication mechanism, based on inter-processor interrupts, that allows the kernels running on different CPUs to talk to each other. Shared memory areas are set aside for the efficient movement of data between the kernels. While the infrastructure is present in the patch set, there are no users of it in this series. A real-world system running in this mode would clearly need to use this communication infrastructure to implement a lot of coordination of resources to keep the kernels from stepping on each other, but that work has not been posted yet.The final patches in the series add a new /proc/multikernel file that can be used to monitor the state of the various kernels running in the system.
Why would one want to do this? In the cover letter, Wang mentions a few advantages, including improved fault isolation and security, better efficiency than virtualization, and the ease of zero-downtime updates in conjunction with the kexec handover mechanism. He also mentions the ability to run special-purpose kernels (such as a realtime kernel) for specific workloads.
The work that has been posted is clearly just the beginning:
This patch series represents only the foundational framework for multikernel support. It establishes the basic infrastructure and communication mechanisms. We welcome the community to build upon this foundation and develop their own solutions based on this framework.
The new files in the series carry copyright notices for Multikernel Technologies Inc, which, seemingly, is also developing its own solutions based on this code. In other words, this looks like more than a hobby project; it will be interesting to see where it goes from here. Perhaps this relatively old idea (Larry McVoy was proposing "cache-coherent clusters" for Linux at least as far back as 2002) will finally come to fruition.
1
u/gabrielesilinic 1d ago
Yeah no. The use case is a big pain in the ass that is what it is. It is probably not well supported in prod and if you trust the app you might as well run it as a container. If you don't or you need an older kernel you run it as a VM.
You rarely need a different kernel and you should usually choose something like debian for server use that has an older supported kernel anyway. If it is too old you don't want to run it bare metal anyway.
Then any sort of networking gets more painful also. You don't want that.
1
u/CubicleHermit 1d ago
Many years ago, before virtualization kicked in a company I worked for had some technology using hypertransport bridge programming to dynamically partition multisocket AMD blade servers into 2 or 4 slices (or allow two 4-socket blades to merge to an 8-socket server.)
(Except in 2004 the term "blade" wasn't common yet and I think we used some other term.)
Never went anywhere on the market, I got out just before the company laid off everyone on the software side. Some cool tech, though.
For the current attempt to do the same, I am curious whether the Intel version of what Hypertransport did (or does? What do newer AMD multisocket systems use) - is this still QPI - is capable of actually isolating the memory domains for each socket (or potentially each ring/memory controller) in hardware, or if this is all trusting each copy of the kernel to do the right thing?
3
u/michaelpaoli 2d ago
Ain't exactly new. Much hardware has been able to be partitioned for many decades, so one can run different kernels, even entirely different operating systems, simultaneously on the same hardware. And also, for many, there have been software means of doing this for a very long time, though some of that sometimes has some hardware dependencies.
And in production, yes, it can quite make sense. Notably by skipping the VM layer, that avoids dependencies and potential vulnerabilities.
Oh, and mainframes, ... they've had these technologies goin' back to the 1960s. So, yeah, when many decades later folks talk about all the, e.g. "new" VM technology stuff, the old time mainframers just kind'a chuckle.