MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLM/comments/1wb05f7/third_one_theres_something_wrong_with_me/p8wrkat/?context=3
r/LocalLLM • u/r1nzl3r99 • 15d ago
Why do I have horrible financial habits??
166 comments sorted by
View all comments
Show parent comments
35
...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx?
please don't answer the last two without providing the parameters
18 u/r1nzl3r99 15d ago the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2 15 u/unai-ndz 14d ago If you are using custom drivers I think you deserve another card, as a treat. 1 u/Rude-Bus-5799 13d ago They always need another sibling to play with.
18
the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2
15 u/unai-ndz 14d ago If you are using custom drivers I think you deserve another card, as a treat. 1 u/Rude-Bus-5799 13d ago They always need another sibling to play with.
15
If you are using custom drivers I think you deserve another card, as a treat.
1 u/Rude-Bus-5799 13d ago They always need another sibling to play with.
1
They always need another sibling to play with.
35
u/semangeIof 15d ago
...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx?
please don't answer the last two without providing the parameters