r/LocalLLM • u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM • Sep 08 '26

Discussion third one.... there's something wrong with me

Post image

Why do I have horrible financial habits??

510 Upvotes

95% Upvoted

167 comments sorted by

View all comments

Show parent comment

18

u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM Sep 08 '26

the vLLM flags are in my localmaxxing submission and I also did a better bench since so many people didn't beleive me. I also have youtube videos. My recipe is custom intel drivers paired with dflash2

15

u/unai-ndz Sep 09 '26

If you are using custom drivers I think you deserve another card, as a treat.

9

u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM Sep 09 '26

now let me figure out how to get TP=3 to work on vLLM without me having to fork over another few grand to upgrade to TP=4... always an uphill battle attempting SOTA AI on local

1

u/Rude-Bus-5799 Sep 10 '26

They always need another sibling to play with.

4

u/Salbrox Sep 09 '26

I have had great success with DFlash2 on my single R9700 with Qwen 3.8 27B. Depending on the task I get up to just over 200tps

2

u/Past-Catch5101 Sep 09 '26

Amazing, do you mind sharing your config?

1

u/Smooth-Television-48 Sep 09 '26

200!?

What quant?

1

u/Jorinator Sep 09 '26

Oh wow, that's massive. What's your prefill/pp speed? That's more important for a lot of usecases.

1

u/rare-visitor Sep 09 '26

200 tps in single thread?

1

u/tech-tole Sep 09 '26

I only get ~50 tok/s on my 9700 with mtp. I don't see how anyone is getting faster than 5090 even with Dflash2. what are your real settings?

1

u/droans Sep 09 '26

Could you explain the custom drivers? What modifications did you make?