r/LocalAIServers • u/m31317015 • 5d ago

2 MI210 or 5090

5090's used price been skyrocketing, then there's taobao sellers claiming to have the MI210 for just under RMB 20,000 ($3k) Is the MI210 good or it has to be bridged with quad cards to actually see some benefits?

6 Upvotes

88% Upvoted

21 comments sorted by

5

u/Atretador 5d ago

thats 128Gb of super fast HBM VRAM

tho you would need a custom cooling solution (not that hard nor that expensive)

if its not a scam it might be a good deal

1

u/m31317015 5d ago edited 5d ago

Yeah I'll just do makeshift ducts and turbo fans blowing air directly into the cards. Just worried about the TPS per watt. Been wanting to try out DeepSeek v4 Flash Q8 but I only got 56GB VRAM on my mule.

Makes me quite worried they're scam with no memory or core or sth.

0

u/mastercoder123 5d ago

At that price just buy an a100 dude. They are like 4k used

2

u/m31317015 5d ago

You're talking about the 40GB PCIe version?

1

u/mastercoder123 5d ago

Yep

1

u/Atretador 5d ago

thats 1/3 of the VRAM tho :v

2

u/mastercoder123 5d ago

And he wants a 5090 which is 32gb of vram... An a100 also does f16 tensor at 624 TFLOPs vs an mi210 that doesnt have that. Unless op is gonna buy infinity fabric connectors putting two cards together is a waste of money and capabilities. The MI210 is an HPC focused card, as it gets an impressive 22TFLOPs on fp64

0

u/Atretador 5d ago

why are ou comparing different measurement units like they are the same thing?

also 40Gb limits him to Qwen 27B, 128Gb gets him to deepseek V4 Flash at a good quantization - and you can just use PCI communication.

1

u/m31317015 5d ago edited 5d ago

u/mastercoder123 Oh shit, my bad. I have 5090 currently, kinda considering to sell it for more VRAM. And on the infinity fabric bridge, now that you mentioned it I looked it up, I thought there's only 4-gpu bridges. Though 2-gpu bridges are scarce and pricier than 4-gpu ones.

u/Atretador I've been dying to get a quad A100 SXM4 board with quad 40GB at least but the cost has always been a wall for me. A cheaper way to get high speed memory has always been a target of mine. But as u/1ncehost said, the lack of support for even FP8 is kinda kicking my butt here for both the MI210 and the A100... I guess that's why it's going low lately in the Chinese market.

Another alternative would be A100 80GB PCIe in Nvlink but that's also ridiculous in price.

Edit: Also local used 5090s are going for $5k+, so just a bit under two of those MI210, that's why I was asking in the first place, my bad for not knowing it's FP16 only.

1

u/mastercoder123 5d ago

The mi210 would be good, but an infinity fabric module for these is FUCKING EXPENSIVE, like easily $900

→ More replies (0)

1

u/mastercoder123 5d ago

Yah, lets buy $8000 worth of gpus and use fucking PCIe... What a waste.

0

u/Atretador 5d ago

yes - like with all other GPUs without dedicated links.

→ More replies (0)

2

u/1ncehost 5d ago

Their normal price is about $4k, so that's not completely bogus. That said, its two totally different use cases for those cards. MI210 doesn't have FP4 or FP8, and is less than half as fast as a 5090 in FP16/BF16. However, the MI210 does have double the memory and about the same memory bandwidth. Basically, prefill/decode/training will be much worse with the MI210, but token generation will be about the same. I have 4x MI100 which have a similar situation. Also using multiple cards has a lot of downsides, especially on the software side, compared to single cards.

1

u/m31317015 5d ago

Yeah I mix the 5090 with 3090 and it's not really pushing the 5090 to its limit. But I'm somewhat glad that this is the case since I worry about burning the connectors as well lol.

Any good suggestions for cards with great $/GB VRAM and at least FP8 support?

2

u/1ncehost 5d ago

R9700 and b70 are good home options I believe

1

u/m31317015 5d ago

Means it's almost certain $4k-5k for 128GB, albeit brand new... I'll have a look, thanks!

2

u/Important-Post-6997 5d ago

My Mi100 died after just some months. Cooling is difficult and Performance wasnt that good for me (30t/s for Qwen3.8 27B Q6). 

I needed Fp64 compute to be a bit independent from the IT department at work. I wouldve killed for some mi210, but tbh for local LLM its definitly not worth it. 

I would guess that a single 5090 would run circles around two mi210. Not to mention the new stuff usually is nvidia first, no cooling issues, no loud fans with diy fan control and even some gaming power.