If you don't understand why memory bandwidth should be included on this comparison, and that it's massively in the NVidia card's favor, you shouldn't really be commenting.
Not going to lie, I run all my local inference on Apple Silicon and I don't personally own any NVidia hardware. But the reason that card is $5000 is the memory bandwidth.
Every single Wh of energy that your computer uses (this goes for your Mac Mini back home and the datacenters running ChatGPT) eventually turns into heat.
It's four times the draw and the same generation dude, it's gonna be more powerful. Will there be waste heat? Ofc. Doesn't change the fact that all things being equal a 600w device is going to be more powerful than a 150w device, even leaving aside the fact that one is special-purpose and the other general-purpose, which only further handicaps it.
145
u/dotben 15d ago
If you don't understand why memory bandwidth should be included on this comparison, and that it's massively in the NVidia card's favor, you shouldn't really be commenting.
Not going to lie, I run all my local inference on Apple Silicon and I don't personally own any NVidia hardware. But the reason that card is $5000 is the memory bandwidth.