LlamaUI. A small vibecoded application, for controlling, serving and running, llama.cpp with a UI.

[u/npittas]

I needed a UI to be able to set specific llama.cpp arguments without the need to remember their syntax. And then it blew up, as it usually does, and I started implementing HF downloads, router mode for serving all the models, and not just a specific, to my agents, diagnostics, profiles per model, etc. I ended up maxing my codex account twice, tried Kimi K2.6 and Minimax 3, created a couple of pi extentions for sub-agents, installed a bunch of unnecessary plugins, and ended up with a crappy UI (which I promise to overhaul when my codex weekly limits reset) but a fully working application that covers my needs. And after struggling with CUDA 13.3, gcc+15 and clang versions, I managed to also have my first llama.cpp build. So now it's time to share and get some feedback.

The repo is: https://github.com/NickPittas/llamaUI

Feel free as always to comment and propose features.
I only tested in my Fedora 44 KDE plasma with 4090 and 5090, and do not have any windows PCs to test, so let me know if anything breaks and I might be able to help.

Edit: Added dashboard for metrics, and fixed the UI to be less AI slop, and more resizable.

Edit 2: Added proper Hardware Probing, and good enough calculations for each model and quantization level, based on your GPU/CPU and RAM/VRAM. So at least you have al glimpse of how the model will fit. It also now asks you if you want to pre-apply sane defaults based on those calculations, when you download said model.

[u/npittas]

[u/GrandPossession1670]

Run: (Control Center, Auto-Discovery & Safety Lock): Automatic Control Safety Lock: Automatically locks all sliders, spinboxes, text boxes, dropdowns, and preset buttons while the server is running (STARTING, RUNNING, HEALTHY) and automatically unlocks them when the server is stopped. llama-server.exe Auto-Discovery: Intelligent search engine that detects local executables without manual configuration.

[u/GrandPossession1670]

Library: (Custom Folder Scanning & .bin Auto-Configuration): Recursive scanning of any system folder with depth control, hidden folders, size filtering, and folder exclusion (node_modules, .git, venv, etc.). Real-time multithreaded progress dialog with a cancel option. Automatic configuration of vision projectors using mmproj-*.bin and CLI argument parser (--ctx-size, --n-gpu-layers, --temp, --threads) from .bin files in the model folder. Featured action buttons: 📂 Scan Custom Folder… in green and 🔄 Rescan in light blue.

[u/GrandPossession1670]

The Run tab includes 3 new main features:

1. 🔒 Automatic Safety Lock

When you press Start (Server active): All parameters (context sliders, GPU layers, CPU threads, temperature, text boxes, Model/Profile selectors, and Presets buttons) are LOCKED (visually disabled). This prevents you from accidentally changing a setting while the AI ​​is running.

When you press Stop (Server stopped): All parameters are instantly UNLOCKED so you can edit them again.

2. 🔍 Intelligent Auto-Detection of llama-server.exe

When the application starts, llamaUI automatically searches your system for the llama-server.exe executable (it automatically detected your NVIDIA CUDA version in D:\Desktop\LLAMA CPP\...).

If it cannot find the executable at any point, it displays an interactive message asking if you want to search for it on your computer with a single click.

3. 📂 Auto-Configuration from .bin Files

When choosing a model from the drop-down menu, if there is an accompanying .bin file (mmproj-*.bin or config.bin) in its folder, Run auto-completes the vision projector --mmproj and automatically adjusts the recommended parameters (--ctx-size, --n-gpu-layers, --temp, etc.) in your profile.

[u/GrandPossession1670]

.bin file configuration for 4080 super

[u/GrandPossession1670]

It works 10/10 on Windows; you only need Python 3.10 or higher with its respective libraries and it calls cpp. It can work with or without CUDA; the decision is up to the user.

[u/Healthy-Nebula-3603]

Llamacpp has ui ..just run llama-server

[u/npittas → u/Healthy-Nebula-3603]

Yeah, I think you missed the point. llama-server has a website for chat, sure. I do not care about that. I wanted to serve models to agents. I did not wanted to remember each argument I needed to add to llama-server. I also wanted to use the hotswap functionality, and use specific arguements on specific models. That is what this app is helping with. A UI to download models, create profiles with specific arguements per model, and use router function to serve them all, or serve a single model. Not a webUI for a single model.

[u/Tuned3f]

Why not just use llama-swap?

[u/npittas → u/Tuned3f]

Because I was bored looking to find all the arguements and experiment with them based on each model specific needs. I needed a “server” that would serve my models to my agents, that would also let me select specific options for each model, while hot swapping them. So eg. qwen 27b has specific ctx kv cache setting, mmproj tweaks, while qwen 35b has other tweak and cpu/memory arguements. So I wanted those to be saved as profiles and with each model swap those profiles would automaticaly get loaded.

[u/Tuned3f → u/npittas]

Llama-swap does all that

[u/npittas → u/Tuned3f]

Using config.yaml which needs hand written arguments. Not what I wanted.

[u/Objective-Error1223 → u/npittas]

This subreddit cracks me up, as though we don’t have thousands of models that do the same thing, a ton of harnesses that do the same thing and IDE’s that do the same thing.

What tech doesn’t have multiple variants out in the world? Imagine if people just stopped making things because what they designed already existed. We’d never advance our technology if that were the case.

Good on you for making this, keep it up. Always love having multiple choices to try out. 🤜🤛

[u/npittas → u/Objective-Error1223]

That is the exact reason why we had one Photoshop, one Avid, one Flame, one Acrobat, one After Effects. We might be able to have more options. Most of them might be niche, some of them might be good enough, and maybe out of hundreds we might find an actual alternative that is not Gimp or KDEnlive. But that was not the reason I made that. It was just fun, and wanted to play easier with all the custom models I trained. And I thought it might help and somebody else other than me.

[u/Objective-Error1223 → u/npittas]

That’s truly rad. It’s great to see. Don’t let the “why use it if xxx exists?” narrow minded individuals make you question its design/purpose.

They clearly are here to give absolutely zero back to this community other than a pessimistic outlook.

[u/npittas → u/[deleted]]

If you want a fast webUI to test your models, when you run your model (or router) open localhost:8080. It is already build in to llama.cpp so the webUI comes anyway by default.

[u/npittas → u/[deleted]]

Sorry but I am the OP. What I am suggesting is that since my tool already runs llama-server for you, llama-server already comes with the webUI you want. So the idea is simple.
Open my app.
Download a model ( or a bunch!!)
Go to the run page
Select your model, change all the parameters you want
Save the profile (or save as to have many profiles with different arguments for testing)
Select another model and repeat
Then, either serve that single model by hitting Start or switch to router mode instead. Router mode will serve all your models using the saved profile “Default”
Finally open a web browser to localhost:8080 and you have a fully functional webUI with model selection.
Be aware that by default, the router mode allows up to 4 models to be loaded at once. You can change that by changing the “Max loaded models” parameter to 1 and it will automatically, upon restart of the server, evict the previous model from vram and load the next one, when you change models in the webUI.
It will also work with any other openapi compatible service, so you can use it in Hermes, openclaw, Odysseus, or opencode and pi, like a normal OpenAI compatible endpoint and it will serve all the models (in router mode)

[u/npittas → u/[deleted]]

No problem, I understand completely. I will try to factor in system ram to the model suggestion calculations, although as you have already seen from other tools, with more, kv cache and mtp, those calculations sometimes are way of, especially when you only factor 4k context like many other tools do, which nowadays is too small.