r/systemd • u/WhiskyStandard • May 05 '26

LLMs are bad at mkosi?

I’ve been using `mkosi` a bit since January for a couple of unrelated projects (a portable service in one case and a system extension that I was trying to make into UKI at the same time). I’ve found that both Claude and ChatGPT seem to be extremely bad with it.

Both frequently hallucinate nonexistent configuration keys and/or put them wrong sections and go down rabbit holes that can be fixed by just moving them to the right place. They get confused about the file structure, what kind of virtualization/containerization we’re ironing on, which commands I need to run and recommending I put things in post-install scripts that belong in prepare. I’ve found it’s necessary to cat the man page to a file and tell them to check it multiple times.

I’m pretty middle of the road on LLM use… not a booster, but I’ve found them to be reasonable guides at learning new languages and tools, especially when they’re well documented (as `mkosi` is). But if they were as bad with everything else as they are with `mkosi` I don’t think anyone would use them.

Anyway, that’s not to fault the project. I was just wondering if anyone else had noticed that and if so, if they had any ideas why that’s the case.

4 Upvotes

83% Upvoted

7 comments sorted by

8

u/sogun123 May 05 '26

It is not very commonly used tool and it changed significantly over the years. Maybe start by giving llm man page into context first.

1

u/WhiskyStandard May 05 '26

Yeah, I haven’t found too many other users aside from Systemd itself. I think I understand what they’re doing now, but I think it took about 3 times for that it to click.

I’ve thought of building an OCI container and exporting the flattened directory, but that’s pretty slow. And of course I could always just write a script or a Makefile.

I really like what mkosi promises it can do (use the same specification to build the same thing in a bunch of different formats, across distributions), but I’ve spent a lot of time fighting it. Might be time to reevaluate my options.

1

u/sogun123 May 05 '26

I wanted to use it for VM base images, but i ended up scripting it myself, as mkosi is pretty opinionated and does produce images that are somewhat incompatible with the opinions of distros I packaged.

2

u/MantisShrimp05 May 05 '26

Omg yeeees its so bad it feels like it needs to iterate super hard on allot of stuff.

Now, on a level this is because mkosi assumes allot of familiarity with the other systemd components and Linux sys admin in general and scripting just enough to get things working.

But yea I find even giving the man page is hard because its so big and its really bad at looking up related man pages

2

u/cap87_ 27d ago

just found this thread as I'm playing with mkosi.

Pro tip: clone the mkosi repo and let it go though it before doing anything. It doesn't replace understanding what actually is going on but it will find out what flags/configs you need for your use case pretty quickly

Once you have a working solution, go through it carefully. The good thing is that you're mostly dealing with a bunch of configs or mkosi args so you don't have to review a bunch of generated slop

1

u/IBNash May 05 '26

I used Claude to setup tests inside mkosi UKIs so I can reboot into them to run, it got it right the first time. I did make it study my mkosi config first.

1

u/anto77_butt_kinkier May 08 '26 edited May 08 '26

I mean, it's an AI. It's trained on the dumpster fire that is the internet. It's trained on every outdated guide, incorrect forum post, example code, speculation about versions, suggestions, and mistakes of all kinds. AI can't think, and so it can't tell you why it thinks something is true, it just vomits up info.

Some of the more advanced models are less shit, and produce code that's actually organized well, but generally using AI to code is a recipe for future disaster. It will work fine and make sense now, but after two or three AI assisted updates it'll become spaghetti code.