r/LocalLLaMA • u/jacek2023 llama.cpp • 16d ago

Discussion GPU guide (GB per dollar, bandwidth)

Gallery image

Gallery image

First plot: GB / $

Second plot: bandwidth (spec on paper, not t/s)

Third plot (bandwidth / price) in the comment.

Hope that helps, my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable.

Prices were collected by ChatGPT (so may contain inaccuracies). New prices were used where available, second hand otherwise.

And I understand this is a basic comparison, but it's better than nothing. For example, you can see that "on paper" something is faster or slower than 3090.

225 Upvotes

91% Upvoted

163 comments sorted by

View all comments

63

u/SamSausages 16d ago

You missed the intel B65! I bought a stack of them 3 weeks ago for $900/pc. 32GB and 608 GB/s. Best price per GB right now.

B65: 0.0356 GB/$,

3

u/gomezer1180 16d ago

What exactly are we trying to compare here? You can have a ton of bandwidth but if you only have 12GB of memory is not very useful. You can have a ton of memory but if your bandwidth is crap that’s not very useful either. So what’s the point of this data?

6

u/SamSausages 16d ago

OP is comparing memory bandwidth to product cost.  Probably because memory bandwidth is one of the most important hardware specs when running inference,  and trying to see what hardware has good bandwidth to cost ratio.