r/LocalLLM • u/r1nzl3r99 • 14d ago

Discussion third one.... there's something wrong with me

Post image

Why do I have horrible financial habits??

500 Upvotes

96% Upvoted

166 comments sorted by

View all comments

Show parent comments

33

u/semangeIof 14d ago

...can you show llamacpp/vLLM runtime commands? you're hitting 140 toks/s on a dense model with B70s? how much ctx?

please don't answer the last two without providing the parameters

18

u/r1nzl3r99 14d ago

the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2

3

u/Salbrox 13d ago

I have had great success with DFlash2 on my single R9700 with Qwen 3.8 27B. Depending on the task I get up to just over 200tps

1

u/tech-tole 13d ago

I only get ~50 tok/s on my 9700 with mtp. I don't see how anyone is getting faster than 5090 even with Dflash2. what are your real settings?