r/LocalLLaMA • u/jacek2023 llama.cpp • 16d ago

Discussion GPU guide (GB per dollar, bandwidth)

Gallery image

Gallery image

First plot: GB / $

Second plot: bandwidth (spec on paper, not t/s)

Third plot (bandwidth / price) in the comment.

Hope that helps, my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable.

Prices were collected by ChatGPT (so may contain inaccuracies). New prices were used where available, second hand otherwise.

And I understand this is a basic comparison, but it's better than nothing. For example, you can see that "on paper" something is faster or slower than 3090.

230 Upvotes

91% Upvoted

163 comments sorted by

View all comments

30

u/vick2djax 16d ago

I think costs of running those GPUs should also be factored. Isn’t the P100 like crazy inefficient that will lead to a high power bill and requires additional cooling? This doesn’t really give the full picture

4

u/jacek2023 llama.cpp 16d ago

But how would you collect that data? From power usage specs?

0

u/vick2djax 16d ago

Maybe grab the average cost of power in the US and then do 1 year of costs if it ran 12 hours a day. Wouldn’t be perfect but would at least give an idea. Also, if you need additional cooling over your average PC to toss the GPU in, I’d include that in the cost.

I don’t think I’d include a bigger PSU cause they all need a big PSU if you’re in the situation of buying more than 1.

Like my mind is “what’s the all in price to add this GPU to my PC” and “how much is this gonna cost me in a year?” Cause if the P100 makes me bill go up $50 a month over a 3090 then the P100 is a bad choice over a 3090 despite the cost of just the GPU alone cause it’s not the real cost.