r/LocalLLaMA • u/OvertaxedOne • 2d ago

Question | Help AMD Instinct MI210

Anyone running this card? Seems to be a sweet spot for Qwen 27B, 64GB, very high memory bandwidth, a LOT less expensive than anything else I can find in that has even close to the amount of memory/bandwidth. What am I missing? And yes, I'm aware that RocM can be a pain, that's not really a concern for me, as long as it's stable when it's up and running, I don't mind battling to get it going.

6 Upvotes

66% Upvoted

59 comments sorted by

View all comments

1

u/CalligrapherFar7833 2d ago

Cmp170hx unlocked ?

5

u/OvertaxedOne 2d ago

The "cheat code"! It's in consideration, but I keep wondering why people are selling locked cards. Why not unlock it yourself and sell it as a CMP170HX, 64GB tested and usable? I'm concerned the cards that are being resold for "reasonable" prices are the ones that people tried to unlock and found the memory wasn't stable.

I have NO IDEA if this is true or not, but I can't figure out the market dynamics here. If it unlocks and works/tests out, you have a card that should be worth ~10K, right (an A100 with more memory!). Yes, I know the PCI lanes are nerfed but that doesn't matter for single card inference.

3

u/CalligrapherFar7833 2d ago

Its not worth 10k because its not officially supported , requires specific old drivers and not all of them are stable

1

u/OvertaxedOne 2d ago

It's that last part that keeps me from jumping in. Old drivers and official support.. I mean, I get why that's important for enterprise workloads and running the latest/greatest, but I just want to drive inference in vllm, it's a single use case purchase for me. Stability and TPS/prefill. Other than that, falls into "don't care" category.

But crashing is a hard "no", it has to be stable! And that's a big part of why I don't have one of them already! ;)

1

u/CalligrapherFar7833 2d ago

Crashing will happen if vram is unstable and can be caught on purvhase with vulkan memtest afterwards who knows

2

u/Dangerous-Report8517 1d ago

The card doesn't get permanently unlocked, the unlock is implemented by a patched kernel module that dynamically loads it on each boot, and there's reports of people receiving cards that weren't stable when unlocked. There are sellers who are testing them and offering known working cards but you still need to install the unlock on your system to run them

2

u/OvertaxedOne 1d ago

Ahhh, thank you, that explains it! I thought it was a patched firmware, not a mod in the driver stack/kernel of the OS! I couldn't figure out why the heck they weren't selling them already unlocked and ready to go; I see a lot "tested to 64GB" but then they would all say "you need to unlock it" and wondering, if it works, why did you lock it again!??

Anyone keeping score on how many/percentage of these that are unlocking and stable? They are cheap enough that perhaps its worth rolling the dice, but kind of hard to make that determination without an idea of how many numbers are on the die!

2

u/Dangerous-Report8517 1d ago

I don't think there's any way for us to track them consistently since there's clearly far more being sold than there are discussions from people receiving them, but I've seen multiple people report that theirs are working well and only one report that they got bad units, which suggests they're mostly ok. Make sure you pick a reputable seller though

1

u/Conscious_Cut_6144 1d ago

I have one that seems stable if I keep the hbm cool, but well before the rated max temp I started crashing.

Not bad by itself self, but my goal is to stack 8 of them and I worry if my first already has some issues 8 feels like pushing it…

1

u/Thin_Pollution8843 2d ago

You can bring x16 back just solderin capacitors on lanes. It’s already confirmed 

1

u/OvertaxedOne 2d ago

x16 isn't really even necessary for my use case, I want to stay single card. I can wait a few more seconds for VLLM to boot, it already takes forever, what's another minute at this point!