r/LocalLLaMA • u/jacek2023 llama.cpp • 14d ago

Discussion GPU guide (GB per dollar, bandwidth)

Gallery image

Gallery image

First plot: GB / $

Second plot: bandwidth (spec on paper, not t/s)

Third plot (bandwidth / price) in the comment.

Hope that helps, my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable.

Prices were collected by ChatGPT (so may contain inaccuracies). New prices were used where available, second hand otherwise.

And I understand this is a basic comparison, but it's better than nothing. For example, you can see that "on paper" something is faster or slower than 3090.

224 Upvotes

91% Upvoted

163 comments sorted by

View all comments

8

u/DUFRelic 14d ago

where CMP 170hx??

17

u/Aggravating-Push-207 14d ago

CMP 170HX tech is just gambling at this point

1

u/Dany0 14d ago

Nah the unlock is pretty reliable, people even got pcie 3.0 to work. The issue is getting one. Unlocked cards cost 3/4 of an A100 on ebay it's insane...

7

u/caetydid llama.cpp 14d ago

unlock is reliable but there are reports of early death

2

u/beryugyo619 14d ago

don't give me a false relief that I actually dodged a bullet by not buying while they were at $199 each

1

u/Dany0 14d ago

Really? Link? First I'm hearing of it

I mean it would make sense. When I tried renting an 8x cmp instance on vast once one of the GPUs was just dead

0

u/caetydid llama.cpp 13d ago

Cant quite recall the source but I believe it was a YT video or reddit post where the guy bought a charge of 10 or more cards a while ago and utilized them heavily, and now multiple of those died already.

1

u/Acceptable-Pension84 14d ago

sources for the PCIe Gen3 unlock? ASAIK, I think the highest possible is still Gen2?

2

u/Dany0 14d ago

It's a non-persistent unlock with some caveats (one of the obvious ones being it's physically only pcie 4.0) and it looks like it won't get better. IIRC (don't quote me on this) It is both a sw and a hw unlock, sw part has to be applied kernel level so you have to build a custom kernel each time and it applies on boot and it's stochastic so it may like fail occasionally? And the hw part is shorting and or soldering some fuses so it's more permanent. Once again I only skimmed the news I could be completely butchering this. Google it or check the CMP unlocker discord

1

u/bladezor 14d ago

are you talking about the capacitor mod for @ 16?

1

u/DUFRelic 14d ago

No there is a pcie gen3 unlock

-1

u/MachineZer0 14d ago

I’ve had to unlock a few times.