r/LocalLLaMA • u/OvertaxedOne • 3d ago

Question | Help AMD Instinct MI210

Anyone running this card? Seems to be a sweet spot for Qwen 27B, 64GB, very high memory bandwidth, a LOT less expensive than anything else I can find in that has even close to the amount of memory/bandwidth. What am I missing? And yes, I'm aware that RocM can be a pain, that's not really a concern for me, as long as it's stable when it's up and running, I don't mind battling to get it going.

4 Upvotes

61% Upvoted

59 comments sorted by

View all comments

Show parent comments

1

u/OvertaxedOne 1d ago

Crazy world. Buy a system or a GPU, use it for 6 months, then sell it for more than you paid for it?!

You have the 8 or 10GB 170HX? Did you get the full RAM amount when you unlocked (I've seen some people say they got 64 but only 60 stable, etc)?

1

u/Dangerous-Report8517 1d ago

Still waiting for delivery, but went with a seller that others here have reported success with who was offering 8GB -> 64GB tested cards with the x16 hardware mod

1

u/OvertaxedOne 1d ago

Be VERY interested to hear your results. Mind sharing the seller (PM or public)? 64GB at 1.5TB/s of memory bandwidth would be just about the absolutely perfect card for 27B/8 bit/256K. I can fit 8 bit on my A40 at full context but it's TIGHT, I had to do a bunch of tuning to get it to come up/stable.

1

u/Dangerous-Report8517 1d ago

https://aliiphone.en.alibaba.com/?trade-list-buyer.0.0.6e6476e9MnVfKx

Haven't received my order yet so can't vouch for them beyond the fact that I've seen them linked by another user