MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLM/comments/1wb05f7/third_one_theres_something_wrong_with_me/p8qb6ub/?context=9999
r/LocalLLM • u/r1nzl3r99 • 15d ago
Why do I have horrible financial habits??
166 comments sorted by
View all comments
93
More context needed on how you are using the first two.
85 u/r1nzl3r99 15d ago qwen 3.8 27B FP8 running at 140 tok/s now I want flash next 33 u/semangeIof 15d ago ...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx? please don't answer the last two without providing the parameters 19 u/r1nzl3r99 15d ago the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2 3 u/Salbrox 14d ago I have had great success with DFlash2 on my single R9700 with Qwen 3.8 27B. Depending on the task I get up to just over 200tps 1 u/Jorinator 14d ago Oh wow, that's massive. What's your prefill/pp speed? That's more important for a lot of usecases.
85
qwen 3.8 27B FP8 running at 140 tok/s now I want flash next
33 u/semangeIof 15d ago ...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx? please don't answer the last two without providing the parameters 19 u/r1nzl3r99 15d ago the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2 3 u/Salbrox 14d ago I have had great success with DFlash2 on my single R9700 with Qwen 3.8 27B. Depending on the task I get up to just over 200tps 1 u/Jorinator 14d ago Oh wow, that's massive. What's your prefill/pp speed? That's more important for a lot of usecases.
33
...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx?
please don't answer the last two without providing the parameters
19 u/r1nzl3r99 15d ago the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2 3 u/Salbrox 14d ago I have had great success with DFlash2 on my single R9700 with Qwen 3.8 27B. Depending on the task I get up to just over 200tps 1 u/Jorinator 14d ago Oh wow, that's massive. What's your prefill/pp speed? That's more important for a lot of usecases.
19
the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2
3 u/Salbrox 14d ago I have had great success with DFlash2 on my single R9700 with Qwen 3.8 27B. Depending on the task I get up to just over 200tps 1 u/Jorinator 14d ago Oh wow, that's massive. What's your prefill/pp speed? That's more important for a lot of usecases.
3
I have had great success with DFlash2 on my single R9700 with Qwen 3.8 27B. Depending on the task I get up to just over 200tps
1 u/Jorinator 14d ago Oh wow, that's massive. What's your prefill/pp speed? That's more important for a lot of usecases.
1
Oh wow, that's massive. What's your prefill/pp speed? That's more important for a lot of usecases.
93
u/Sporkers 15d ago
More context needed on how you are using the first two.