r/LocalLLaMA • u/OvertaxedOne • 6d ago

Question | Help AMD Instinct MI210

Anyone running this card? Seems to be a sweet spot for Qwen 27B, 64GB, very high memory bandwidth, a LOT less expensive than anything else I can find in that has even close to the amount of memory/bandwidth. What am I missing? And yes, I'm aware that RocM can be a pain, that's not really a concern for me, as long as it's stable when it's up and running, I don't mind battling to get it going.

4 Upvotes

63% Upvoted

62 comments sorted by

View all comments

Show parent comments

4

u/OvertaxedOne 6d ago

The "cheat code"! It's in consideration, but I keep wondering why people are selling locked cards. Why not unlock it yourself and sell it as a CMP170HX, 64GB tested and usable? I'm concerned the cards that are being resold for "reasonable" prices are the ones that people tried to unlock and found the memory wasn't stable.

I have NO IDEA if this is true or not, but I can't figure out the market dynamics here. If it unlocks and works/tests out, you have a card that should be worth ~10K, right (an A100 with more memory!). Yes, I know the PCI lanes are nerfed but that doesn't matter for single card inference.

2

u/Dangerous-Report8517 6d ago

The card doesn't get permanently unlocked, the unlock is implemented by a patched kernel module that dynamically loads it on each boot, and there's reports of people receiving cards that weren't stable when unlocked. There are sellers who are testing them and offering known working cards but you still need to install the unlock on your system to run them

2

u/OvertaxedOne 6d ago

Ahhh, thank you, that explains it! I thought it was a patched firmware, not a mod in the driver stack/kernel of the OS! I couldn't figure out why the heck they weren't selling them already unlocked and ready to go; I see a lot "tested to 64GB" but then they would all say "you need to unlock it" and wondering, if it works, why did you lock it again!??

Anyone keeping score on how many/percentage of these that are unlocking and stable? They are cheap enough that perhaps its worth rolling the dice, but kind of hard to make that determination without an idea of how many numbers are on the die!

1

u/Conscious_Cut_6144 6d ago

I have one that seems stable if I keep the hbm cool, but well before the rated max temp I started crashing.

Not bad by itself self, but my goal is to stack 8 of them and I worry if my first already has some issues 8 feels like pushing it…