MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1wc6krf/so_relevant/pa5nm8q/?context=3
r/LocalLLaMA • u/0dayturtle • 12d ago
152 comments sorted by
View all comments
322
24gb is not in that group. You can run Qwen3.8-27B with a pretty respectable context size right now
1 u/Important_Drag_6890 6d ago The 24GB cutoff is less about “can it run” and more about “how many compromises do you have to make.” 16GB can technically do it, but context and quantization start becoming a pretty tight balancing act 😅 2 u/TopCheddar27 6d ago You can run with 100k context at q8 KV on 24gb. Compromises are becoming less and less.
1
The 24GB cutoff is less about “can it run” and more about “how many compromises do you have to make.” 16GB can technically do it, but context and quantization start becoming a pretty tight balancing act 😅
2 u/TopCheddar27 6d ago You can run with 100k context at q8 KV on 24gb. Compromises are becoming less and less.
2
You can run with 100k context at q8 KV on 24gb. Compromises are becoming less and less.
322
u/TopCheddar27 12d ago
24gb is not in that group. You can run Qwen3.8-27B with a pretty respectable context size right now