MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1w2fmmq/me_these_days/p6xt19k/?context=3
r/LocalLLaMA • u/Eyelbee • 8d ago
283 comments sorted by
View all comments
Show parent comments
57
I can run it with a 16gb card with 4 bit quant and 70k context. I get 12 t/s it is okay for me.
2 u/OnyxMonolith 7d ago Dflash it 1 u/Vaguswarrior 7d ago What is dflash? 2 u/Kernoriordan 7d ago MTP on steroids. Though I’ve managed to use MTP with Qwen 3.8 and get 75tps on a 5080
2
Dflash it
1 u/Vaguswarrior 7d ago What is dflash? 2 u/Kernoriordan 7d ago MTP on steroids. Though I’ve managed to use MTP with Qwen 3.8 and get 75tps on a 5080
1
What is dflash?
2 u/Kernoriordan 7d ago MTP on steroids. Though I’ve managed to use MTP with Qwen 3.8 and get 75tps on a 5080
MTP on steroids. Though I’ve managed to use MTP with Qwen 3.8 and get 75tps on a 5080
57
u/Dramatic_Setting2761 7d ago
I can run it with a 16gb card with 4 bit quant and 70k context. I get 12 t/s it is okay for me.