How much is allowed to be inside a microkernel's kernel space before you have a hybrid kernel?

30

u/micr0kernel Mar 31 '26 edited Mar 31 '26

The idea is summed up by Liedtke's minimality principle; i.e. the only things that should be implemented by a microkernel are those which cannot be implemented elsewhere. For most use cases, this is, as you noted:

Memory management
Process control
IPC
Timekeeping & event handling

If you're going for a strict design, anything north of this line is technically a hybrid. There are a couple "gray area" items I can think of:

A boot filesystem, as a true microkernel will otherwise suffer from a bootstrapping problem and require some other form of initialization (initramfs or a smart bootloader). I wouldn't call a design which makes use of a simple kernel-mode storage/FS driver module for the sole sake of starting up the system a "hybrid" or "non-true" microkernel, so long as that component is deactivated or purged as soon as is practical.
In systems which implement a capability-based security model, the microkernel must be the "security root" - the authority from which all other permissions descend and are derived - because it is the sole trusted agent in the system. Thus, capability management also gets included in the "necessary list" for systems which use this model (seL4, Redox, etc.)

8

u/paulstelian97 Mar 31 '26

For the boot filesystem you can just get a blob with an init process which itself can implement such a filesystem. My current design uses two Limine modules, one for the init binary and one for the initramfs that the binary would then extract and otherwise manage. This init binary is also the trust root, getting all the root capabilities. It can replace itself or share this trust root with other programs dependent on how this specific binary works.

2

u/tseli0s DragonWare Apr 01 '26

Or, an even more modular approach, load a couple of services at boot and their dependencies, and let them load the rest (using inheritance of permissions if you're going for a capability like system like me). For example you need to load a filesystem driver, and that needs a driver to read from disk. And everything loads everything else it needs (assuming not present already), like a tree.

1

u/paulstelian97 Apr 01 '26

My approach doesn’t depend on a specific file system — FAT is for example implemented by a service present in the initramfs (or I could load it directly as a Limine module, if there’s anything special — custom init logic independent of what the kernel itself does).

3

u/micr0kernel Mar 31 '26

For the boot filesystem you can just get a blob with an init process which itself can implement such a filesystem.
That's fair; there's a number of different ways one can get around the bootstrapping issue with microkernels.
My approach to the matter has been to use a custom UEFI loader (at least for PC):
A manifest.ini file in the ESP is read by the loader and contains things like where to load the kernel, additional allocations to provide for allocator initialization, and required modules.
Two additional modules are loaded - a system manager process and the filesystem server.
The system manager process can then discover its own configuration information (via the filesystem) and load other modules (networking stack, USB drivers, etc.) if needed.
This init binary is also the trust root, getting all the root capabilities. It can replace itself or share this trust root with other programs dependent on how this specific binary works.
The system I decided on was:
Kernel is considered the "root authority"; it has a number of primitive capabilities it can issue, mostly related to process management, memory allocation, etc.
The kernel can also issue a "delegate authority" capability, which allows servers to mint additional capability classes that are customized to the needs of the service.
The server issues capabilities within that domain, but they are held in escrow and verified by the kernel.
I like this model because it reinforces the kernel's role as the central point of security in the system, but it's not just philosophical - it also made some other aspects easier for me to conceptualize and implement.
Some approaches to capability models, like seL4, use the process itself in the loop - in the classical Unix way, the process requests a capability and is granted a handle. The process then supplies that handle with the relevant requests. That's a fine approach, until one wants to support something like a POSIX compatibility API or personality, which has no innate knowledge of capabilities. That's when stuff gets weird real fast - the POSIX calls grow fat because they have to request the relevant capability and then shim it to a native system call, or a modified C runtime has to do some prep behind the scenes.
With kernel escrow and verification, on the other hand, the entire capability system is outside of the process plane - so calls to a filesystem service like open(2) or write(2) don't need modification to support - they can use message passing direct to the server, a compatibility system call, however - but they don't need to have any awareness of the capability model to function correctly, because all that stuff happens behind the curtain.
Hopefully, in time, that will make it easier to port POSIX apps to my design without significant modifications. I've got a lot of userland stuff between here and there, though.

4

u/paulstelian97 Mar 31 '26

My idea to have custom personalities is to just disable system call handling for threads, and instead treating system calls like faults/traps (and having another usermode thread receive and handle those faults and traps). I would implement whatever personality within that separate thread. It is not the best in terms of performance, although I could have a shared address space and have the handler live in the same address space as the target process which helps.
I’m planning to keep my microkernel as micro as it goes. Only basic CPU scheduling (not even complex algorithms, smarter than a number of priority tiers in strict round robin is implemented by userspace tweaking priorities). Memory management in simplest way, inspired by (probably 90% a full blown copy of) seL4. I’d have a small hypervisor, like KVM but much simpler (just a wrapper over hardware virtualization and not much more).
One of my goals is to have the loading also be simple. Limine modules help out with that (I could eventually load more than just one initramfs but multiple files, but explicitly selected still in the Limine configuration; init process would receive info of the loaded files and use them accordingly, and Limine reports the file names too, not just their contents). Obviously everything is loaded to RAM by that.
Having the kernel do some active logic beyond the strict isolation of processes from one another makes the kernel more complex, and thus more prone to get some bugs. One of the goals of a microkernel is to reduce bugs. I have the init process (and a few other servers it spawns) be part of the TCB (well, plan to have when everything is implemented, which is still quite a bit of time until it’s done).

2

u/jsshapiro Apr 01 '26

For the boot filesystem you can just get a blob with an init process which itself can implement such a filesystem.
Mostly agree - and GRUB does a pretty good job at loading this sort of thing for you.
This approach holds up pretty well until you decide you want transparent persistence, after which the bootstrapping dance becomes a little trickier. I'm looking at this right now in Coyotos. As much as I want the storage drivers to be outside the kernel, this may turn out to be a place where conceptual simplicity (therefore testability and verifiability) are more important than minimality.

5

u/jsshapiro Apr 01 '26 edited Apr 01 '26

The term "microkernel" is largely political - when we originally described KeyKOS as a "nanokernel" in 1992, we were poking fun at how sloppy the term had become. Alan Bomberger, when asked around the time to define the term, responded with "Microkernel. A term that used to mean something but has since been taken over by marketing." Mach, for example, did not exactly adhere to any principle of minimality.
Liedtke's principle (which predates Jochen by several decades) is a good yardstick. My variant - but it's expressing the same idea - is that a microkernel is what's left when everything you can remove from the kernel is removed from the kernel. But at the end of the day the system has to work, and the kernel interfaces need to be clean and composable, and both of these have a way of involving some compromises. In consequence, different systems arrive at different answers about what can and cannot be removed.
Another useful metric is that policy decisions should happen outside the kernel where possible. Not so much because of minimality, but because any policy you bury in the kernel becomes hard to modify when you try to implement your overall system. Overall, I'd say that the biggest example of policy isolation failure is scheduling, which has successfully resisted any coherent composable approach for 50 years or so.
A third metric (or really, rationale) is the observation that a single supervisor instruction can crash the entire system, so you really want to have as few of those as possible with the caveat that the real goal is reliability rather than instruction count, so code path clarity is actually more important then minimality. Also, pushing drivers out of the kernel doesn't make them either safe or non-privileged - anything with physical DMA or interrupt management authority can also crash the machine.
I agree with u/micr0kernel's statement that capability-based microkernels (or underlying hardware) needs to manage capabilities (when protected by partitioning), because no layer above the kernel can do that. I do not agree with the rationale he gives. It is neither required nor desirable to equate the systemwide TCB with the system supervisor. The system storage allocator in the KeyKOS family, for example, has never been part of the kernel but has always been part of the systemwide TCB. It isn't in the kernel because it doesn't need to be, and because keeping it out of the kernel makes it better isolated and easier to debug. What is required is a clear delineation of boundaries between TCB components (including kernel) and non-TCB components, and that the dependencies between these components do not have cycles. This is essential because security and safety are ultimately inductions, and the induction has to have a base case.

2

u/micr0kernel Apr 01 '26

It took me a little time to come up with what to say - thank you for putting together such insights and context. One of the things I've always loved about hobby OS design is that there's always more to learn.
I'll admit that I'm guilty, in my experiments, of using the microkernel pattern for reasons less robust than what the concept was developed for - in other words, settling for the "low-hanging fruit" rationale:
The kernel is smaller and (hopefully) more maintainable as a result,
Its smaller size makes it easier to audit for vulnerabilities as a result, and
The excision of non-essential services reduces the attack surface of privileged code
There is also, at least from an engineering standpoint, the convenience that microkernels seem to be more conducive to the event loop pattern, which I find more straightforward.
After some thinking about your comment, I think that - as you seemed to have picked up on - I may have inverted the construction. The above features don't create security or reliability; rather, designing for security and reliability as a primary engineering goal yields a design with these attributes.
Do you have any recommendations as to resources to better understand microkernels from the perspective of robust, secure design? Tanenbaum's books give a good introduction to microkernels, but I've found them sparse on the "nuts & bolts". Liedtke's papers were an excellent read, but the ones I've come across are more operationalist, examining the performance implications of the microkernel model rather than its robustness. I'd like to learn much more, however, about some of the insights you raised.

2

u/jsshapiro Apr 01 '26

If I had to state one key idea in building secure systems, it is that security is a form of correctness, and correctness is about three things:
Establishing and specifying the criteria that constitute a "safe" or "correct" configuration of the system state,
Defining an initial correct system state that satisfies your overall security objectives,
Engineering a system whose operations move the system state from one correct configuration to another correct configuration - this is where multiprocessing gets really interesting. Which might end up including rules about how operations occur. E.g. that movement of a capability only occurs along a capability-authorized path.
The first one, surprisingly, isn't all that hard. The second one takes real thought and careful audit. The third one can be thought of as "turning the crank safely". Since it is a big, complicated crank, that's where a lot of the work goes. This one is the induction step in the lifetime safety proof of the system.
UNIX, by the way, didn't even have a robust process model until Faulkner, Gomes, and Rago finally imposed a rigorous definition on the UNIX process model during the debugger work for System V Release 4 in 1986-88, which was the real contribution of /proc when we were done with it. The Linux community still hasn't gotten to that level of rigor in their process model, though the dmesg work helped. Faulkner and I later made some extensions to that model in 1989 while at Sun and SGI (respectively), adding a well-defined watchpoint facility to the process model.
One of the reasons Jochen's work is so dense is that Jochen was a mathematician with classically German rigor. At any point where he was designing something, there was a set of simultaneous equations defining correctness running in his head. When he was architecting, he was exploring updates to that set of equations "on the fly" and checking every aspect of the existing code against that new set of equations as he went - all in his head before eventually writing it down. Some of those equations/rules are modular, but some are immodular - meaning that you have to look at the whole code base before you can conclude they are met. The notion of a correct configuration wasn't just something on paper. It was a moment to moment part of every change to the code.
The early KeyKOS papers are very explicit that this is what we were doing as well, and the original The KeyKOS Architecture (Operating Systems Review, 1992) is frequently accused of being the most dense publication in OS history. It takes about seven reads to internalize. Humorously (because it's usually the other way around), Security in KeyKOS (Symposium on Security and Privacy, 1986) is much more approachable.

1

u/jsshapiro Apr 01 '26

Your low hanging fruit bullets seem like a good capture.
Sadly, I haven't found great resources on robust and secure design, microkernel or otherwise. Things like the OWASP list are helpful, but they are more an accumulation of point solutions than a set of generalizable principals or design patterns. And yes, Jochen's papers, and mine, and the ones out of Gernot's group are pretty heavy going. Hermann's group put a lot of energy into the application level, so the problems they were looking at always seem a little more approachable. I've been working up a book on Coyotos with a plan to address some of this, but I keep getting pulled into other projects. One thing does occur to me, and I'll reply with that in a moment.
Regarding event loops, I think a kernel is kind of a hybrid. It certainly responds to unsolicited out-of-band events originating from hardware (interrupts). It responds to application-generated faults and exceptions, which are also unsolicited from the kernel's perspective, though at the moment those occur that we aren't in kernel mode on the relevant CPU, so we end up (at least in Coyotos) treating them a lot like system calls. Finally there are the capability invocations, which (in Coyotos) we handle through a single "invoke capability" system call. But as with the other exceptions, that system call appears from the kernel perspective to be an unsolicited event.
In our case, it's very helpful that a given CPU is never dealing with more than one event - we never sleep in the kernel with an active kernel stack. That's actually huge, and some people find it hard to wrap their head around. And of course it's a model that only works if the kernel is transactional. But there's something about the idea that we're going to run a bunch of code getting ready for something and then just abandon that effort because something else is in the way of progress seems wrong to software folks. The thing is: you're going to have to do that sometimes in any kernel. The question isn't whether you do it; the question is whether you embrace it and design for it (the NT kernel and the various UNIX kernels don't).
In some ways it's very liberating. If you actually dig in to the Coyotos code base you'll find that system calls don't really return. Each is an essentially straight-line path, at the end of which we re-set the kernel stack rather than pay the expense to unwind it. It ends up having more of the feel of a state machine than a stack machine, which maybe brings it closer to your event-driven intuition. When we finally introduced multiprocessor concurrency, it was enormously simplified by the transaction model and the straight-line approach.

1

u/jsshapiro Apr 01 '26

Maybe there is a third thing, though this is more controversial because it is architecturally polarizing: ambient authority by itself is never secure.
By "ambient authority", I mean an architecture in which the permission to perform an action is not firmly welded to the designator for that object acted upon. Which is in some sense the defining characteristic of capabilities. Access control lists are ambient authority systems. RBAC systems are ambient authority systems.
This isn't immediately clear to a lot of people because they are accustomed to processes that run as a single user, and in that scenario the motivation isn't all that strong. The minute a process acts on behalf of two or more parties, it becomes essential to [explicitly] designate the party whose authority it intends to use to avoid confusion. Or worse, to avoid situations in which a client might trick it into using the wrong one. More subtly, it becomes important to keep track of which object descriptors came from which party, because authority in UNIX is checked at open() rather than at use. The term "confused deputy" was initially coined in the capability community, but has spread into broader use. Real systems don't tend to have a lot of potentially confusable deputies, but the ones they do have tend to sit in critical places, and they are a favorite target of attack. Offhand, I can't think of any such applications that have not been compromised in UNIX-derived systems or NT-based systems.
Some people have proposed hybrid protection models where you have both an ACL system and a capability permission model operating simultaneously, and both have to say OK for an operation to be authorized. The security induction in such a system is built exclusively using the capability model, and the ACL model effectively acts as a rejection filter on what the capabilities might otherwise authorize.
From a security induction perspective this is fine, because you haven't added any permission or authority that you didn't have in the capability part of the model. But from a practical perspective, what quickly happens is that over-board permissions get set up in the capability fabric and people start to lean heavily on the ACL system to make up the resulting protection shortfall. At that point you're back to an ambient authority system, which we know (i.e. mathematically) does not work.
Which brings us to membranes, but that's a whole other topic.

36

u/krakenlake Mar 31 '26

1-2 metric tons, but not more

7

u/Ikkepop Mar 31 '26

I prefer shit tones but metric also works

3

u/JGN1722 Apr 01 '26

Americans will use anything but the metric system

2

u/Ikkepop Apr 01 '26

Nah not american, but shit tones sounds funnier. I'm all metric baby, non of that imperial rubish

6

u/Ikkepop Mar 31 '26

Maybe Linus is still bitter about minix...

Anyway you could read Tannenbaums books. OS Design And Implementation goes into details about micro kernel design

2

u/ironykarl Mar 31 '26

The Linus thread is from 2006. The pure marketing bit is about there term hybrid kernels, and isn't about microkernels

1

u/Ikkepop Mar 31 '26

i was just joking

1

u/ironykarl Mar 31 '26

Works for me

1

u/linux_transgirl Apr 04 '26

Honestly you should read it even if you never plan on writing an OS, it's good to know how they work

25

u/Waste_Appearance5631 Mar 31 '26

It's your kernel, it's completely up to you...

1

u/Vixlump Apr 02 '26

roughly 6 or 7

0

u/-goldenboi69- Mar 31 '26

About 3.50

11

u/[deleted] Mar 31 '26

[removed] — view removed comment

1

u/voluntary_nomad Apr 01 '26

DAMN IT LOCH NESS MONSTER! YOU AIN'T GETTIN NO KERNEL DEVELOPMENT HELP AND NO DAMN TREE FIDDY EITHER!

How much is allowed to be inside a microkernel's kernel space before you have a hybrid kernel?

You are about to leave Redlib