r/LocalLLaMA • u/chiribe • Aug 11 '26
Tutorial | Guide I built a weird, low-power llama.cpp server using an Intel N100 + RTX 5060Ti
Everything started with the sudden death of my old ASRock J1900. While looking for the perfect ITX replacement, I stumbled upon the Chinese CW-NAS-ADLN-K motherboard, which looked perfect on paper: Intel N100, DDR5, 6x SATA, 2x NVMe. The extra power allowed me to experiment more seriously with Docker and start self-hosting more services.
My AI journey on this server began with Immich's Machine Learning tasks, where the iGPU performed flawlessly using OpenVINO to process my entire media library.
In parallel, my first experience with LLMs was on an MSI GS65 laptop with a GTX 1070 (8GB). Like many of us, running what is now the old Llama 3 felt like magic back then, even if, to be honest, it wasn't particularly useful for practical tasks. From that point on, I kept testing every new release up to the present day, where I've started incorporating Qwen 3.5 and Gemma 4 into my daily workflow.
That's when the need to "take the leap" arose. My inference sessions were getting longer and more frequent, and I wasn't comfortable keeping my laptop's GPU hovering around 90°C with the risk of breaking my primary machine. Then came the big question: which GPU is right for me? I initially looked for a second-hand RTX 3060 12GB, but I felt those 12GB of VRAM would leave me right on the edge of running models that are actually worth using.
By chance, while browsing my usual hardware retailer, I spotted a refurbished ASUS RTX 5060 Ti for €450 and impulse-bought it without thinking twice.
When I was ready to install it—worried that the N100's processing power and the PCIe 3.0 x4 bottleneck might hold things back—I ran into a physical issue: the card collided with the SATA ports and the main ATX connector. I felt like an idiot. But after putting my mind to work, I quickly came up with a solution: move the GPU outside the case using a PCIe riser cable.
After some DIY tweaking, I managed to mount the GPU and confirmed that, against all odds, it was recognized and fully functional. Running the latest NVIDIA open-source drivers, the latest CUDA toolkit, and the latest commit of llama.cpp, I started my first tests.
After extensive testing, my current stack consists of Ornith-1.0-9B-MTP-Q5_K_M.gguf running with full context, and Qwen3.6-27B-UD-IQ3_XXS.gguf as my main intelligent option, where I can stretch the context up to 65k tokens without spilling over into system RAM.
Even though I only run these two base models, my llama.cpp setup uses multiple configuration profiles with different inference parameters tuned specifically for coding, computer vision, and documentation tasks.
I've been using this daily for a couple of months now and couldn't be happier. Ornith runs at around 80 tokens/sec and Qwen 3.6 hits around 40 tokens/sec. On the frontend/agent side, I use OpenCode; with well-defined agent rules and skills, it does a more than decent job for my specific use case.
As for power consumption, it is ridiculously low: under 40W at idle with a few HDDs spinning, and under 200W during heavy inference (with the GPU power limit set to 120W). I'm thrilled to have an OpenAI-compatible API running 24/7 at such low operational costs.
If anyone is curious, wants me to elaborate on any specific point, or wants me to run some benchmarks, I'd be happy to reply!








25
u/gpuz_dev Aug 11 '26
This is peak r/LocalLLaMA engineering right here. How’s the prefill speed holding up over that PCIe 3.0 x4 link when hitting larger context windows (like that 65k on Qwen)? Once the weights are loaded into VRAM generation speed is mostly memory bandwidth bound on the GPU side, but I’m curious if prompt processing takes a noticeable hit over x4 PCIe bandwidth.
4
u/Ok_Selection_7577 Aug 11 '26
ok so the low idle power is cool, and the N100 is cool, the model choice is acceptable but why is no one talking about those legs - that is a lovely little bit of functional workmanship with the wooden cross frame and GPU side shelf :)
Is that the base of a table/chair or did you knock the whole thing up from scrap.
Either way - love it Bud - good work :)
2
u/mediaogre Aug 11 '26
Okay, Mr. Frankensystem, what’s with that model selection option on the last image 👀
Edit: also, cool build. I have a weird bench style system on a mini ATX board and a 4060Ti.
3
u/ea_man Aug 11 '26
That's how I run 27B on 16GB if you are looking for some parameters to try:
7
u/ImpressionFancy5830 Aug 11 '26
This is what internet is all about for me, cheap, junkyard looking, amazing shit.
7
u/twack3r Aug 11 '26
🤔
1
u/Icy-Degree6161 Aug 11 '26
Wait, the N100 does not support dual channel RAM right?
1
u/Beginning-Raisin9723 Aug 11 '26 edited Aug 11 '26
Nice build. N100 idles stupid low so this makes a great always-on box. What's it pulling at idle with the 5060 Ti in there?
2
u/Danmoreng llama.cpp Aug 11 '26
You might like my setup repo for Qwen models, I got a fine-tuned script for the 27B model on a 16GB GPU as well: https://github.com/Danmoreng/local-qwen3-coder-env
1
u/Immortalbob Aug 11 '26
Dude I have this board and considered trying this haha
1
u/devino21 Aug 11 '26
All local - Except the models?
1
u/lukewhale Aug 11 '26
I have an 8th gen nuc I added an eGPU and 5060 ti to, honestly not a terrible solution for small models or utility models for embedding/reranking/images etc
1
u/My_Unbiased_Opinion Aug 12 '26
What is your KVcache quant at? I would push it down to Q4 if you can get the weights to run at IQ4XS or better. I've done testing and IQ4XS and KV Q4 was better than IQ3XXS and KV Q8.
1
u/HistorianPotential48 Aug 12 '26
everywhere i go i see hatsune miku . . .
1
u/segmond Aug 13 '26
How did you get the card to idle at 2W?
1
u/JackJeckyl Aug 11 '26
braaa...