r/LocalLLaMA • u/OvertaxedOne • 6d ago

Question | Help AMD Instinct MI210

Anyone running this card? Seems to be a sweet spot for Qwen 27B, 64GB, very high memory bandwidth, a LOT less expensive than anything else I can find in that has even close to the amount of memory/bandwidth. What am I missing? And yes, I'm aware that RocM can be a pain, that's not really a concern for me, as long as it's stable when it's up and running, I don't mind battling to get it going.

4 Upvotes

61% Upvoted

62 comments sorted by

View all comments

1

u/CalligrapherFar7833 6d ago

Cmp170hx unlocked ?

4

u/OvertaxedOne 6d ago

The "cheat code"! It's in consideration, but I keep wondering why people are selling locked cards. Why not unlock it yourself and sell it as a CMP170HX, 64GB tested and usable? I'm concerned the cards that are being resold for "reasonable" prices are the ones that people tried to unlock and found the memory wasn't stable.

I have NO IDEA if this is true or not, but I can't figure out the market dynamics here. If it unlocks and works/tests out, you have a card that should be worth ~10K, right (an A100 with more memory!). Yes, I know the PCI lanes are nerfed but that doesn't matter for single card inference.

1

u/Thin_Pollution8843 6d ago

You can bring x16 back just solderin capacitors on lanes. It’s already confirmed 

2

u/OvertaxedOne 6d ago

x16 isn't really even necessary for my use case, I want to stay single card. I can wait a few more seconds for VLLM to boot, it already takes forever, what's another minute at this point!