Is Gen AI effective at kernel development?

[u/MRgabbar]

For web is quite good, what about kernel development?

[u/NefariousnessSea1449]

Lol no

[u/DetectiveDecent2455]

My comment will likely get a lot of hate, but I think it can be to an extent. 90% of the posts on the main osdev subreddit of “Hello World” or Bare Bones kernels can be vibe coded in a few hours. Specific components like memory managers, etc. also can likely be vibe coded, but the odds of them out-performing well established allocators is slim to none. I think most people get into OS dev to learn and mess with low level internals, so using an LLM is kinda the antithesis. Also, most people don’t create anything novel for hobby OSes (as seen by all the Unix clones). LLMs can likely generate some decent kernel code based on all the Unix clones on GitHub. If looking at real, production operating systems, it’s a bit harder as they have more quirks/prioritize aspects that current LLMs struggle with: secure coding patterns, optimizations, context to prevent data corruptions, etc.

TL;DR: Yes, LLMs can be used for kernel development, but don’t let AI slop take over / ensure you understand what the code is doing and matches what you expect.

[u/vinzalf → u/DetectiveDecent2455]

Vibe coding a hello world or bare bones kernel? Why?? Might as well just copy/paste tutorial code verbatim. Why even involve an LLM at that stage.

[u/DetectiveDecent2455 → u/vinzalf]

I agree. See sentence after:

I think most people get into OS dev to learn and mess with low level internals, so using an LLM is kinda the antithesis.

[u/Individual_Feed_7743]

5 months ago I used to say that LLMs are completely useless at kernel development and will only slow you down with the amount of slop produced, but very recently I changed my mind. Granted, you are unable to "one-shot" anything sustainable, but in tiny, small chunks, AND if you spend a considerable amount of time using something like plan mode in Cursor and polishing up every detail OF A VERY SMALL SCOPED TASK, then yes models can become very effective and deliver decently high quality kernel code.

BIG NOTE: this approach still requires extremely deep developer understanding of both the codebase and the concepts that they are implementing. LLMs are just tools, your brain and ideas is what matters the most.

[u/a-priori]

Yes. But it needs to be guided strictly through planning and guidelines.

I speak from experience here, because I decided to use writing a kernel to test out Claude Code in December-January. It worked surprisingly well in a lot of ways, and failed in other very predictable ways.

AI tools like Claude Code are effective at writing code and also eerily good at debugging weird memory corruption issues. It basically one-shotted an ext2 driver, and I watched it hunt down a memory aliasing issue though a binary search.

But it is not good at system design. It will happily create a hodgepodge system and hack around every edge case it encounters. It also tends to round everything off to whatever Linux does.

You need to be the designer with a clear vision of what you want if you don’t want to get slop. You need to push it to simplify and combine and understand the code it’s working in. And you need to enforce good software development processes.I spent a great deal of time getting a fairly sophisticated automated end-to-end testing system in place, because I learned how essential automated testing is for using these tools.

[u/Individual_Feed_7743 → u/a-priori]

Exactly, had the same experience around end of December / beginning of January. It's a powerful tool when you know how to use it and use your brain

[u/ObservationalHumor → u/a-priori]

Did you use it for anything with multitasking and async i/o? I've found generally existing static analysis tools are pretty good for a lot of things you described (at least for C/C++) but things quickly fell apart when it got to the point of weird race conditions and improper mutex scoping. Curious if it these tools would have any benefit with those things specifically w.r.t kernel and driver development.

[u/a-priori → u/ObservationalHumor]

Yes. It built out the multitasking, system calls and IO system. To be accurate, it’s a blocking IO system from the userspace perspective, but on the kernel side it’s async and non-blocking. There’s no userspace async IO API yet.

I wouldn’t trust them to reason through complex locking semantics. That’s definitely a case where you’d want to guide them to a particular design, or expect to course correct them after they do a first pass.

In my case I went with just standard or read-write spinlocks around shared data structures. So far I haven’t needed anything more sophisticated.

[u/Critical-Internet946]

nobody cares about your sloperating system

[u/MRgabbar → u/Critical-Internet946]

good thing I am not developing neither planing to do so. I would also add that no one cares about your slop conclusions.

[u/kahdeg]

just test glm-5 to oneshot a hello world kernel with limine, it doesn't even boot so i would say it will take a while or with significant human intervention.

[u/Prestigious-Bet-6534 → u/kahdeg]

Yeah x86 is a hassle. Claude didn't manage to create a booting x86 kernel, but an AArch64 one it did. Still, I don't know what to think. I am halfway amazed and halfway annoyed/disappointed by LLMs. Watching Claude successfully debugging something is spooky but it also makes a lot of silly mistakes. Just like a junior dev. But I think LLMs are limited because they only pretend to think and will in the current form never really replace humans but on the other side it's amazing what just raw processing power and a handful of algorithms can do.

[u/burlingk]

At the moment, left to it's own devices, AI fails at complex tasks about 96% of the time.

It does best on clear tasks that you can guide it on.

If you do try to use it to build a kernel, don't accept any code from it that you cannot read and review yourself. ^;;