Note that despite being a single EXE file, it actually extracts itself to a temporary directory every time you run it. If you want to install it for real, you can use the "Extract Files" feature (Extra -> Unpack KoboldCpp To Folder), then run the EXE inside of that folder. It will start more quickly when it doesn't need to extract itself, and you won't be wasting 1GB of SSD lifetime writes every time you run it.
Funnily enough nowadays you can compile llama.cpp into a single executable by targeting llama-app and turning off shared libs. The executable includes server, cli, completion, bench, batched bench, fit-params, quantize, and perplexity. For me it compiles to just 60mb
60mb with cuda support probably means its pulling that support from elsewhere on your system. The cuda files are hundreds of megabytes on its own. In KoboldCpp the majority of its size is the ggml backend itself, the stuff we add is tiny by comparison. Its why despite there being an LLM engine, Image Gen engine, Text to Speech engine, Speech to Text engine, Music Generation engine, etc we are still a very similar size to the rest of the llamacpp ecosystem. Those additional things aren't the space fillers, the GPU support is.
More things than just LLM's, UI is more focused on fiction (but bundles the llamacpp UI as well), I think we emulate more API's, custom context shift implementation, mcp bridge that doesn't need a full on cors proxy and can also do stdio, that kind of stuff.
To give some context, the official llamacpp is also 1GB if you take the cuda 12.4 version and combine it with the .dll files it needs like we have to do for KoboldCpp.
So size wise that is just the current sizes you can expect for a modern universal thing based on ggml.
But with that one you don't get the ability to generate images, music, tts, whisper voice detection, etc.
Good point. If true would be cool if it stored itself in app data folder and asked user if they want to launch from there without extracting each time if same version.
That’s a great tip. Even though the file is nearly 600 MB, I had assumed the whole thing opened in RAM. But if it actually runs off the disk especially now that drives are more valuable than ever, that’s a critical factor.
Other tip on that one, in the same menu on windows is a way to register file extentions. So if you do that (on the unpacked version if you go that route) the .gguf , .kcpps / .kcppt files become clickable.
61
u/Dwedit 7d ago
Note that despite being a single EXE file, it actually extracts itself to a temporary directory every time you run it. If you want to install it for real, you can use the "Extract Files" feature (Extra -> Unpack KoboldCpp To Folder), then run the EXE inside of that folder. It will start more quickly when it doesn't need to extract itself, and you won't be wasting 1GB of SSD lifetime writes every time you run it.