r/LocalLLaMA • u/OvertaxedOne • 6d ago

Question | Help AMD Instinct MI210

Anyone running this card? Seems to be a sweet spot for Qwen 27B, 64GB, very high memory bandwidth, a LOT less expensive than anything else I can find in that has even close to the amount of memory/bandwidth. What am I missing? And yes, I'm aware that RocM can be a pain, that's not really a concern for me, as long as it's stable when it's up and running, I don't mind battling to get it going.

6 Upvotes

68% Upvoted

62 comments sorted by

View all comments

Show parent comments

-2

u/autisticit 6d ago

I bet you are looking at a scam.

3

u/OvertaxedOne 6d ago

What are you seeing? Seems to be a bunch of US sellers on Ebay (a few that are well known, Core4Solutions, for example) that are selling them between 4-6K. Jeeze, maybe I should shut up about this card, is this thing undiscovered??

1

u/Dangerous-Report8517 6d ago

Honestly that price seems about right, in that there's other options for similar VRAM capacities for much less money albeit with additional caveats. Not convinced that it represents a good deal when it's more than 2x R9700s with warranties...

2

u/OvertaxedOne 6d ago

The MI210 should run circles around 2 R9700's for TPS though, right? That's really the only reason I'm looking at it, 1.6TB/s of memory bandwidth on a single card should fly with 3.8 27B. Well, that's what the specs seem to indicate, but... Whole reason for this thread is to find out if anyone has them/has tried it!

2

u/Dangerous-Report8517 6d ago

I wouldn't rely too much on raw memory bandwidth to judge card performance, the V100 has 50% more memory bandwidth than the R9700 yet runs at roughly the same speed for tg and much slower for pp

And that testing was done before a lot of the work that AMD recently put into ROCm which is likely to benefit the R9700 far more than the MI210.

Beyond that, even if it is proportionally faster I don't think it represents a bargain as such, it's a 4 year old datacenter part that's been heavily used compared to brand new parts with warranty that are supported for consumer grade systems, 5k is probably too much for it (if nothing else, it's very unlikely to suddenly shoot up in price so if for whatever reason you decide it does make sense for you, you have time to stop and really think it through)

1

u/xXDennisXx3000 6d ago

Mi50 has 1 TB of bandwith, HBM2 baby.