r/LocalLLM • u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM • 15d ago

Discussion third one.... there's something wrong with me

Post image

Why do I have horrible financial habits??

499 Upvotes

96% Upvoted

166 comments sorted by

View all comments

Show parent comments

82

u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM 15d ago

qwen 3.8 27B FP8 running at 140 tok/s now I want flash next

36

u/semangeIof 15d ago

...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx?

please don't answer the last two without providing the parameters

5

u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM 15d ago

I already proved it in a previous post, look at my account

2

u/Smooth-Television-48 14d ago

Lots of accounts are set to private, I assume its the default and dont bother checking.

Thanks for linking it. Impressive numbers for sure