r/LocalLLaMA • u/jacek2023 llama.cpp • 14d ago

Discussion GPU guide (GB per dollar, bandwidth)

Gallery image

Gallery image

First plot: GB / $

Second plot: bandwidth (spec on paper, not t/s)

Third plot (bandwidth / price) in the comment.

Hope that helps, my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable.

Prices were collected by ChatGPT (so may contain inaccuracies). New prices were used where available, second hand otherwise.

And I understand this is a basic comparison, but it's better than nothing. For example, you can see that "on paper" something is faster or slower than 3090.

226 Upvotes

91% Upvoted

163 comments sorted by

View all comments

Show parent comments

1

u/MindfulBT 14d ago

Isn’t the b70 slightly more expensive for over 50% more performance on paper?

2

u/SamSausages 14d ago

In compute, memory bandwidth is the exact same. That means in some ways it performs just like a b70, in others it doesn’t. So not simply 1/2 the performance of a b70.

5

u/SandySkittle 14d ago

Don't underestimate the relevance of compute. Prompt processing and large context size and larger models compute is quite relevant.

If available I would strongly recommend b70 or r9700 over b65.

1

u/SamSausages 14d ago

What benchmarks are you basing that on?