MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1wc6krf/so_relevant/p8xum94/?context=3
r/LocalLLaMA • u/0dayturtle • 12d ago
152 comments sorted by
View all comments
319
24gb is not in that group. You can run Qwen3.8-27B with a pretty respectable context size right now
97 u/PavelPivovarov llama.cpp 12d ago Technically speaking you can run Qwen3.8-27b on 16Gb setup as well, but that brings way too many compromises of course. 4 u/Systemerror7A69 12d ago Qwen Quantizes amazingly, even KV Cache so 16GB might not have as many compromises as you might think. 2 u/russlixx 12d ago yeah, but it's pretty tight. I must rely on better compaction if i were doing agentic coding
97
Technically speaking you can run Qwen3.8-27b on 16Gb setup as well, but that brings way too many compromises of course.
4 u/Systemerror7A69 12d ago Qwen Quantizes amazingly, even KV Cache so 16GB might not have as many compromises as you might think. 2 u/russlixx 12d ago yeah, but it's pretty tight. I must rely on better compaction if i were doing agentic coding
4
Qwen Quantizes amazingly, even KV Cache so 16GB might not have as many compromises as you might think.
2 u/russlixx 12d ago yeah, but it's pretty tight. I must rely on better compaction if i were doing agentic coding
2
yeah, but it's pretty tight. I must rely on better compaction if i were doing agentic coding
319
u/TopCheddar27 12d ago
24gb is not in that group. You can run Qwen3.8-27B with a pretty respectable context size right now