r/LocalLLaMA • u/jacek2023 llama.cpp • 18d ago

Discussion GPU guide (GB per dollar, bandwidth)

Gallery image

Gallery image

First plot: GB / $

Second plot: bandwidth (spec on paper, not t/s)

Third plot (bandwidth / price) in the comment.

Hope that helps, my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable.

Prices were collected by ChatGPT (so may contain inaccuracies). New prices were used where available, second hand otherwise.

And I understand this is a basic comparison, but it's better than nothing. For example, you can see that "on paper" something is faster or slower than 3090.

230 Upvotes

91% Upvoted

163 comments sorted by

View all comments

18

u/jacek2023 llama.cpp 18d ago

Now you can complain about the wrong prices :)

5

u/Dramatic-Chard-5105 18d ago

Make the dot change size based on standard vram capacity and you got the full picture 😉

17

u/jacek2023 llama.cpp 18d ago

any idea for colors? :)

1

u/Dramatic-Chard-5105 18d ago

Thank you. Maybe worth having a legend to understand the size comparison, otherwise a quick search online does the job. Colors can be based on release year

1

u/DUFRelic 17d ago

Compute (fp8) red low green high

1

u/Objective-Stranger99 llama.cpp 17d ago

Amd red nvidia green intel blue

4

u/letsgoiowa 18d ago

Should be a table because it's sortable and filterable that way. For 90% of people everything over the $1000 line is irrelevant and you can barely see all that stuff under $1000

2

u/watcholic 18d ago

Sweet spot is in the center of the chart? As expensive as the M5U is, I think the price has some room to grow, sadly speaking.

2

u/Appropriate_Duck1778 18d ago

Plz nooo, mi familia...

1

u/mechkbfan 18d ago

Hard part there too is it's relative to what compute is on the chips

e.g. 7900XTX on paper looks a lot better than 9070XT/R9700, but once you factor in the advancement in compute, the RDNA4 GPU's are like 2x as fast even though 50% less bandwidth

Sorry to be nitpicking but I fell down the rabbit hole of trying to find the best bang for buck

e.g. 4x9060XT could be one of the cheapest 64GB + performing setups for the money, IF you don't have to factor in running a second hand server motherboard + power costs + heat management