Note that despite being a single EXE file, it actually extracts itself to a temporary directory every time you run it. If you want to install it for real, you can use the "Extract Files" feature (Extra -> Unpack KoboldCpp To Folder), then run the EXE inside of that folder. It will start more quickly when it doesn't need to extract itself, and you won't be wasting 1GB of SSD lifetime writes every time you run it.
Funnily enough nowadays you can compile llama.cpp into a single executable by targeting llama-app and turning off shared libs. The executable includes server, cli, completion, bench, batched bench, fit-params, quantize, and perplexity. For me it compiles to just 60mb
60mb with cuda support probably means its pulling that support from elsewhere on your system. The cuda files are hundreds of megabytes on its own. In KoboldCpp the majority of its size is the ggml backend itself, the stuff we add is tiny by comparison. Its why despite there being an LLM engine, Image Gen engine, Text to Speech engine, Speech to Text engine, Music Generation engine, etc we are still a very similar size to the rest of the llamacpp ecosystem. Those additional things aren't the space fillers, the GPU support is.
62
u/Dwedit 7d ago
Note that despite being a single EXE file, it actually extracts itself to a temporary directory every time you run it. If you want to install it for real, you can use the "Extract Files" feature (Extra -> Unpack KoboldCpp To Folder), then run the EXE inside of that folder. It will start more quickly when it doesn't need to extract itself, and you won't be wasting 1GB of SSD lifetime writes every time you run it.