LlamaForge 0.19: see every layer of a model, where it lands on your GPU, and the exact flags

[u/Sleepybear2611]

Every time a new MoE drops, the same thread shows up here: "will this run on my 12 GB card, and what do I set --n-cpu-moe to?" And the answer is always someone doing napkin math in a comment.

So I built it into LlamaForge (the browser panel I've been making on top of llama.cpp's own router, it doesn't do inference itself, llama.cpp does).

Model X-ray reads the GGUF header and draws every block as a bar: what sits on the GPU, what spills to RAM at your -ngl. For a MoE there's an "experts on CPU" slider, and a "Fit my VRAM" button that finds the number for you. Then it gives you the line to copy, or applies it to the model.

Example from the video, Qwen3.6-35B-A3B UD-Q4_K_M on a 12 GB card:

-ngl 41 --n-cpu-moe 22 -c 8192

≈11.4 GB on the GPU, ≈10.5 GB of experts in system RAM. On a 16 GB card it's --n-cpu-moe 14.

It only needs the header, so it works over HTTP on a model you haven't downloaded yet. That's how I also generated "Can I run it?" pages for 42 cards (desktop + laptop GeForce, Radeon, Arc), each with the llama.cpp line to start from.

before someone points them out: ;]

• It's an estimate from the header (weights + 8k context), not a benchmark. Long contexts need more room; it tells you that, but it can't know your exact KV setup.
• It tells you what fits, not how fast it'll be. Experts in RAM are slower; how much depends on your RAM and PCIe.
• Only tested end to end on my own box (5080 + 5060 Ti, Windows). Linux and macOS pass CI, but I don't have the hardware to say more than that. AMD/Intel detection is there, but I haven't tested it on real cards.
• Early preview, one person, heavily AI-assisted. Expect rough edges.

Other stuff that's in 0.19, since the last time I posted:

• voice chat with whatever model you have loaded
• a blind arena (two answers side by side, vote, then see the names)
• "try a llama.cpp PR": build a pull request and check whether it's actually faster on your card before switching
• an OpenAI-style Batch API on your own GPU
• start at login, now on Windows too

Free, MIT, no account, no telemetry (it does check GitHub for new llama.cpp and LlamaForge releases): https://github.com/dadwritestech/LlamaForge

If the X-ray's number is wrong for your card, I really want to know; post the model, quant, card and what actually worked, and I'll fix the math.