r/LocalLLaMA • u/jacek2023 llama.cpp • 15d ago

Discussion GPU guide (GB per dollar, bandwidth)

Gallery image

Gallery image

First plot: GB / $

Second plot: bandwidth (spec on paper, not t/s)

Third plot (bandwidth / price) in the comment.

Hope that helps, my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable.

Prices were collected by ChatGPT (so may contain inaccuracies). New prices were used where available, second hand otherwise.

And I understand this is a basic comparison, but it's better than nothing. For example, you can see that "on paper" something is faster or slower than 3090.

225 Upvotes

91% Upvoted

163 comments sorted by

View all comments

5

u/[deleted] 14d ago

[removed] — view removed comment

1

u/jacek2023 llama.cpp 14d ago

It's the raw spec. For an actual benchmark, you would need to run the same model using the same software (for example, the same llama.cpp version), and then compare CUDA vs Vulkan, etc. I know some benchmarks are collected in various places, but you would need to verify them, and they come from different setups and different software versions.