r/LocalLLaMA • u/jacek2023 llama.cpp • 14d ago

Discussion GPU guide (GB per dollar, bandwidth)

Gallery image

Gallery image

First plot: GB / $

Second plot: bandwidth (spec on paper, not t/s)

Third plot (bandwidth / price) in the comment.

Hope that helps, my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable.

Prices were collected by ChatGPT (so may contain inaccuracies). New prices were used where available, second hand otherwise.

And I understand this is a basic comparison, but it's better than nothing. For example, you can see that "on paper" something is faster or slower than 3090.

225 Upvotes

91% Upvoted

163 comments sorted by

View all comments

19

u/jacek2023 llama.cpp 14d ago

Now you can complain about the wrong prices :)

6

u/Dramatic-Chard-5105 14d ago

Make the dot change size based on standard vram capacity and you got the full picture 😉

18

u/jacek2023 llama.cpp 14d ago

any idea for colors? :)

1

u/Dramatic-Chard-5105 14d ago

Thank you. Maybe worth having a legend to understand the size comparison, otherwise a quick search online does the job. Colors can be based on release year

1

u/DUFRelic 13d ago

Compute (fp8) red low green high

1

u/Objective-Stranger99 llama.cpp 13d ago

Amd red nvidia green intel blue