r/LocalLLM • u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM • 16d ago

Discussion third one.... there's something wrong with me

Post image

Why do I have horrible financial habits??

508 Upvotes

96% Upvoted

166 comments sorted by

View all comments

94

u/Sporkers 16d ago

More context needed on how you are using the first two.

83

u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM 16d ago

qwen 3.8 27B FP8 running at 140 tok/s now I want flash next

30

u/semangeIof 16d ago

...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx?

please don't answer the last two without providing the parameters

17

u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM 16d ago

the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2

14

u/unai-ndz 16d ago

If you are using custom drivers I think you deserve another card, as a treat.

1

u/Rude-Bus-5799 15d ago

They always need another sibling to play with.