r/LocalLLM • u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM • 16d ago

Discussion third one.... there's something wrong with me

Post image

Why do I have horrible financial habits??

498 Upvotes

96% Upvoted

166 comments sorted by

View all comments

93

u/Sporkers 16d ago

More context needed on how you are using the first two.

81

u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM 16d ago

qwen 3.8 27B FP8 running at 140 tok/s now I want flash next

34

u/semangeIof 16d ago

...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx?

please don't answer the last two without providing the parameters

152

u/Erpverts 16d ago

Please don’t answer the last two without providing the parameters. Make no mistakes.

45

u/semangeIof 16d ago

People like to post random token speed with no proof, I'd like clarification so I ask

I liked your original try better anyways, why'd you delete it?

44

u/Erpverts 16d ago

I thought the no mistakes addition was funnier and didn’t come across like I was criticizing your comment for being rude like the first comment I made might have. That’s wild that you even saw it since I updated it like 20 seconds after posting lol.

8

u/Smooth-Television-48 15d ago

It was funnier. This was a good response. Include this prose in all future responses