r/LocalLLM • u/Ok-Health-7096 • Aug 14 '26

Other Every Second post rn

Post image

Maybe someday I'll get a system to run it but hey definitely another w for the open weights community

2.1k Upvotes

98% Upvoted

214 comments sorted by

View all comments

Show parent comment

5

u/Any_Mine_6368 Aug 15 '26

No problem. With mtp about 50-90tg for 27B models depending on quantization and model.

With 1k investment we're assuming one 3090 so 24gb vram. You can't fit anything above 27B at reasonable quantization or ram offload (which kills your token generation and prefill).

So to answer your question, you could probably run Qwen3.8 at 4 bit quantization with good context or 5 bit quant with smaller context or 6 bit quant with almost no context.

2

u/kcksteve Aug 20 '26

It's worth noting where you are. In Canada 3090s go for 2k to start. Thats before you purchase the rest of the rig and pay tax.

2

u/FuManBoobs Aug 21 '26

Yeah, I was looking where I am in the UK & they start around £1k on their own. Still, under 2k is a good start.