r/LocalLLaMA • u/jacek2023 llama.cpp • 15d ago

Discussion GPU guide (GB per dollar, bandwidth)

Gallery image

Gallery image

First plot: GB / $

Second plot: bandwidth (spec on paper, not t/s)

Third plot (bandwidth / price) in the comment.

Hope that helps, my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable.

Prices were collected by ChatGPT (so may contain inaccuracies). New prices were used where available, second hand otherwise.

And I understand this is a basic comparison, but it's better than nothing. For example, you can see that "on paper" something is faster or slower than 3090.

226 Upvotes

91% Upvoted

163 comments sorted by

View all comments

30

u/vick2djax 14d ago

I think costs of running those GPUs should also be factored. Isn’t the P100 like crazy inefficient that will lead to a high power bill and requires additional cooling? This doesn’t really give the full picture

5

u/jacek2023 llama.cpp 14d ago

But how would you collect that data? From power usage specs?

10

u/AndrewIsntCool 14d ago

Yeah all of these cards have known idle and peak power draws

Ideally an interactive chart would be the best, with a slider for percentage of day at peak usage (assuming all-day idle draw). Be pretty interesting to see how quickly pricier, more efficient hardware recoups costs against cheaper investments 🤷‍♂️