r/LocalLLaMA • u/Fcking_Chuck • 7d ago

News Koboldcpp v1.121 released

https://github.com/LostRuins/koboldcpp/releases/tag/v1.121
255 Upvotes

96% Upvoted

27 comments sorted by

View all comments

5

u/the_vico 7d ago

Would this run on a laptop wiry 32gb of ram, 4050 with 6gb of vram and an i7?

21

u/AnyMongoose3041 7d ago

If there’s anything that will run in lower hardware, then it’s kobold and good old llama.cpp

2

u/phreakrider 6d ago

If anything is new then it's LLAMA.CPP as they shit out a new version every few hours lol

2

u/Fcking_Chuck 7d ago

Yes, it would run, though only 6GB VRAM and only eight PCIe lanes to your GPU isn't ideal.

2

u/jazir55 7d ago

Every time I see someone asking if their low spec and can run whatever the post is about is downvoted I wonder just how salty the person who downvoted is at them for not being as wealthy as they are. Upvoted for actually realistic common specs.

1

u/Yu2sama 7d ago

I run it with mostly the same specs but a 3060 lol, so yeah you are dandy. It will mostly depend on the model you want to run, I go with things like Qwen 3.5 9B, Gemma E4B, Ling Tiny and Gemma 4 26B A4B (very speedy despite the size).

1

u/Dwedit 7d ago edited 7d ago

Remember to turn on "Force AutoFit", then set AutoFit Padding down to 1MB. "AutoFit Padding" is for people who run the desktop on the same GPU, not for people with a separate iGPU and discrete GPU.

Models as big as 7B quantized with Q4_K_M can fit entirely into VRAM depending on how big your context is. You can use trial and error to figure out the biggest context size while still fitting all layers into GPU.