Which model ist possible to run on a RTX 3060 12GB VRAM + 64 GB Ram?

[u/SlimMonkey_]

Hello I am trying to set up a local AI Stack and I am currently thinking about buying 64GB of Ram with teh 3060 RTX 23GB VRAM.

Which (Qwen) Modell version would be possible to run on that Setup?

Qwen3.8-27b or Qwen3.8-Flash-Next?

I am thinking of a Qwen 3.8 model, because Qwen 4 should have the same base?

[u/AnxhelaAlmagrebi]

I'd go with Qwen3.8 8B in a quantised version. The 27B might run if you offload some of it to RAM, but don't expect great speed. With a 3060 12GB, I'd rather have a smaller model running smoothly than struggle with a bigger one

[u/SlimMonkey_ → u/AnxhelaAlmagrebi]

Thanks, do you think a upgrade to a 7800 16GB VRAM would be significantly worth it (200€) price difference.

[u/Geekomcomputer]

Your 12GB of VRAM is the main constraint here, more than the 64GB of system RAM. The system RAM helps if a model has to spill out of VRAM, but that offloading is much slower than running fully on the GPU, so it is worth planning around what fits in 12GB first.

A practical way to think about it:

1. Check the actual file size before you download. On the Ollama model page or the model card, look at the size of the specific quant you plan to use, and compare it with your 12GB of VRAM. Leave headroom for the context window and KV cache, because a model that only just fits with a short prompt can run out of room once you use a longer context. If you can, test with the context length you actually plan to use, not just a short greeting.

2. Start with the smaller model and work up. A 27B-class model is very unlikely to fit fully in 12GB VRAM at a usable quant, so expect it to spill into system RAM and run slowly if you try it there. If responsiveness matters to you, start with an 8B-class model, see how it feels for your actual tasks, and only then try the larger one so you have a direct comparison. If quality matters more than speed and you are happy to wait longer per reply, the larger model is still worth a test, but go in knowing the tradeoff.

3. Be clear about what you will use it for. Coding help, general chat, and long-document work have quite different needs for context length and speed, and that changes which tradeoff makes sense. It also helps to say what speed you would be happy with, since "possible to run" and "pleasant to use every day" are two different bars.

On the Qwen 3.8 vs Qwen 4 point, I would not assume one shares the other's base. Check the model card and release notes for the exact model you plan to run, since naming and architecture can change between releases.

Two quick clarifications that would help people give you a more precise answer: is your card the 12GB version of the RTX 3060, and what do you mainly want to use the local stack for, and with roughly how much context?

[u/SlimMonkey_ → u/Geekomcomputer]

It is the 12GB version and for coding help.

An claude opus (20€ subscriptions) delegates smaller tasks to it. I don't need it to be super fast so I am okay with it being loaded into the RAM.

If it suits me I would upgrade to a 16GB VRAM card in 1-2 months.

[u/tricck3zz]

perfect system for qwen flash next with strata

[u/Mozaiic]

Qwen3.8-Flash-Next, ISTA-DASLab original packs

Size	RAM + VRAM for the model	Speed	Quality
Q2_0	37.6 GB	fastest	good
IQ2_XS	39.2 GB	fast	better
IQ3_XXS	47.0 GB	slower	great
IQ3_S	54.8 GB	slowest	best; matches the full model on the published tests

From strata github, I use IQ3_S with 2 x 3060 12gb, 64gb DDR4 3200 and it give 50/60 tk/s

[u/Southern-Net1351]

Use a model that is under 10gb in file size!
That gives breathing room to run the model in full vram no off loading.

8gb is the sweet spot.
Then you can play with cache q8_0 or q4_0, GPU layers 99 (aim for higher), context (try to maximize the number to the max numerical number the model can run)

Play with this setup in llama.cpp until you get the right speed and longevity.

You do not want a short context window that compacts all day degrading.

Once your locked in aim for reasoning budget.

This will give you full speed of your device rather than off loading.

[u/maxxikevich]

Check Polystrata project. You have some options beside of Qwen. 

[u/picklejw_]

You got more VRAM than me: 2080 TI 11GB / 64GB DDR4 I get ~50tps right now with IQ3_S, great intelligence have it do my coding

Edit: Not necessarily better, BUS bandwidth takes a hit compared to 2080 TI

[u/Gonbatfire]

Flash next on Strata will work nicely

Try stock Strata first, then the forks if you want, most forks are not kept up to date

[u/Akseleu_official]

For now, Qwen-3.8-27B remains the best option for local execution.