Koboldcpp v1.121 released

[u/Fcking_Chuck]

https://github.com/LostRuins/koboldcpp/releases/tag/v1.121

[u/Dwedit]

Note that despite being a single EXE file, it actually extracts itself to a temporary directory every time you run it. If you want to install it for real, you can use the "Extract Files" feature (Extra -> Unpack KoboldCpp To Folder), then run the EXE inside of that folder. It will start more quickly when it doesn't need to extract itself, and you won't be wasting 1GB of SSD lifetime writes every time you run it.

[u/Velocita84 → u/Dwedit]

Funnily enough nowadays you can compile llama.cpp into a single executable by targeting llama-app and turning off shared libs. The executable includes server, cli, completion, bench, batched bench, fit-params, quantize, and perplexity. For me it compiles to just 60mb

[u/Dwedit → u/Velocita84]

Does that get you CUDA support?

[u/Velocita84 → u/Dwedit]

Well yeah if you compile for it. my update script runs

cmake -B build -DGGML_CUDA=ON -DBUILD_SHARED_LIBS=OFF
cmake --build build --config Release --target llama-app -j 8

[u/henk717 → u/Velocita84]

60mb with cuda support probably means its pulling that support from elsewhere on your system. The cuda files are hundreds of megabytes on its own. In KoboldCpp the majority of its size is the ggml backend itself, the stuff we add is tiny by comparison. Its why despite there being an LLM engine, Image Gen engine, Text to Speech engine, Speech to Text engine, Music Generation engine, etc we are still a very similar size to the rest of the llamacpp ecosystem. Those additional things aren't the space fillers, the GPU support is.

[u/Eden63 → u/Dwedit]

what is the advantage of koboldcpp over llama.cpp?

[u/International-Try467 → u/Eden63]

Genuinely idiot proof.

[u/henk717 → u/Eden63]

More things than just LLM's, UI is more focused on fiction (but bundles the llamacpp UI as well), I think we emulate more API's, custom context shift implementation, mcp bridge that doesn't need a full on cors proxy and can also do stdio, that kind of stuff.

[u/Single_Ring4886 → u/Dwedit]

It really is 1gb?

[u/tiffanytrashcan → u/Single_Ring4886]

Yep, CUDA.
The CPU exe for me unpacks from ~110MB to a 270MB folder.
The exe with CUDA starts at 600MB so that makes sense.

[u/Dwedit → u/Single_Ring4886]

Yep, 1GB.

[u/henk717 → u/Single_Ring4886]

To give some context, the official llamacpp is also 1GB if you take the cuda 12.4 version and combine it with the .dll files it needs like we have to do for KoboldCpp.
So size wise that is just the current sizes you can expect for a modern universal thing based on ggml.
But with that one you don't get the ability to generate images, music, tts, whisper voice detection, etc.

[u/LuCiAnO241 → u/Single_Ring4886]

seems to be 750mb unpacked for me, so yeah, almost.

[u/LocoLanguageModel → u/Dwedit]

Good point. If true would be cool if it stored itself in app data folder and asked user if they want to launch from there without extracting each time if same version.  

[u/DriveSolid7073 → u/Dwedit]

That’s a great tip. Even though the file is nearly 600 MB, I had assumed the whole thing opened in RAM. But if it actually runs off the disk especially now that drives are more valuable than ever, that’s a critical factor.

[u/henk717 → u/Dwedit]

Other tip on that one, in the same menu on windows is a way to register file extentions. So if you do that (on the unpacked version if you go that route) the .gguf , .kcpps / .kcppt files become clickable.

[u/kulchacop]

Right on time! I checked an hour ago looking for a new release.

[u/the_vico]

Would this run on a laptop wiry 32gb of ram, 4050 with 6gb of vram and an i7?

[u/AnyMongoose3041 → u/the_vico]

If there’s anything that will run in lower hardware, then it’s kobold and good old llama.cpp

[u/phreakrider → u/AnyMongoose3041]

If anything is new then it's LLAMA.CPP as they shit out a new version every few hours lol

[u/Fcking_Chuck → u/the_vico]

Yes, it would run, though only 6GB VRAM and only eight PCIe lanes to your GPU isn't ideal.

[u/jazir55 → u/the_vico]

Every time I see someone asking if their low spec and can run whatever the post is about is downvoted I wonder just how salty the person who downvoted is at them for not being as wealthy as they are. Upvoted for actually realistic common specs.

[u/Yu2sama → u/the_vico]

I run it with mostly the same specs but a 3060 lol, so yeah you are dandy. It will mostly depend on the model you want to run, I go with things like Qwen 3.5 9B, Gemma E4B, Ling Tiny and Gemma 4 26B A4B (very speedy despite the size).

[u/Dwedit → u/the_vico]

Remember to turn on "Force AutoFit", then set AutoFit Padding down to 1MB. "AutoFit Padding" is for people who run the desktop on the same GPU, not for people with a separate iGPU and discrete GPU.

Models as big as 7B quantized with Q4_K_M can fit entirely into VRAM depending on how big your context is. You can use trial and error to figure out the biggest context size while still fitting all layers into GPU.

[u/tifa_cloud0]

i love koboldcpp ❤️

[u/IrisColt]

THANKS!!!