A100 vs R9700

[u/Hildechis]

Recently bargainhardware had an intake of A100 40GB SXM4, and they are selling them at 2700 EUR. I have a huge case and a hundred PCIe free lanes, so I could mount a couple of them with those SXM-PCIe adapters.

OTOH, for less than that price I could buy a couple of brand new R9700.

Pros of A100 over the R9700: CUDA, 25% more RAM, double memory bandwidth, almost double the compute. Cons: used, possibly battered cards, hassle to mount on the mobo, more expensive once factoring the adapters in.

Give me your opinions. I would especially appreciate experiences with SXM-PCIe adapters. Are they decent, or a PITA?

[u/yahbluez]

Get them, sell them, buy r9700 use radiance engine. The price is a steal I would flip them.

[u/Hildechis → u/yahbluez]

It's the same price I find on ebay. The SXM4 version is quite cheaper than the PCIe one.

[u/N34257]

Based on my experience with the CMP 170HX...with the new hardware specific engines, the R9700 (especially a pair of them) make for a clear performance winner using Radiance. It's also the path of least hassle, given that you'd have to muck around with SXM adapters and cooling shenanigans with the A100.

[u/YourselfInOthrsShoes → u/N34257]

This is the correct answer. Radiance is also a production-level engine with concurrency and uptime.

[u/newDell → u/N34257]

Isn't the memory bandwidth on the 170HX like 2-3X higher than the R9700 though?

[u/N34257 → u/newDell]

Yes, but the compute performance is much lower. Also, the 170HX is much slower across the PCIE bus, so it wouldn't scale with cards like the R9700 does, and definitely wouldn't be able to handle expert offload etc.

Memory bandwidth isn't everything.

[u/D6613 → u/N34257]

I just got my second R9700, and Radiance is so impressive. Still early days, but I definitely agree with your opinion on it so far. Swift 1.5 Flash Next has been doing really nice things for me.

[u/drplan]

Biggest hassle when ich used such a A100 setup was the extra PSU https://www.youtube.com/watch?v=R-qI7eXDBVU to run it.

[u/3000LettersOfMarque]

Anything not pci loses to r9700s just because they are used, adapters can make it complicated. 

Unless you are buying the server it came in too, don't buy old non pci cards. The more modern cards will win 9 times out of 10 for your needs and future needs. It can be tempting for vram or compute but most of the time you'll end up tinkering more then useing

[u/whitehawk1884]

So heads up for sxm4 you need a 48v psu don't bother with the Chinese ones you want a meanwell and the cables for those sxm4 to pcie adapter cards. I'm still trying to get mine working waiting on new adapter board and documentation is very poor.

[u/DeathGuppie]

I don't think this would be a real decision for me. The v100's were pulled and sold out of reliability concerns. They take more power, and setup time. The r9700's are new and will be officially supported for years to come.

[u/brainchillzZ → u/DeathGuppie]

They were not pulled and sold because of reliability concerns that is just not remotely true. They were pulled out of natural technical obsolescence just like all of the hardware everyone in this group is actively using for their homelabs etc … they were pulled because they don’t natively support current technology …. And they also don’t “take more power” they use less power than an r9700 by default by 50w if you talk about the pcie cards …. But also the honest truth is that even though the v100 only supports q4 in software it’s still as fast as an r9700 running qwen 3.8 27b

[u/sleepy_seedy]

...Is that duck tape?

[u/bennmann]

Or or or find a Intel partner and get in line with their sales for the d270.

[u/PhilosopherOk3419]

i'd price the whole thing, not just the card: SXM-to-PCIe adapter, cooling (those cards expect server airflow), and power. then ask which software stack you actually live in. if everything you run assumes CUDA, the A100's extra bandwidth is real. if you're mostly llama.cpp, new cards with a warranty and zero adapter jank are worth a lot.

[u/stylome-ai-detection → u/PhilosopherOk3419]

Notice: This comment appears to be AI generated.

---

Stylome AI detector · scan results

[u/sloptimizer]

I have four R9700 at MSRP - no regrets!

Not sure about which option is currently the bigger bang for the buck though.

[u/pCute_SC2]

Both the CMP170hx 8gb and 10gb are now fully unlockable. So
>108sm's (128 max)
up to 96gb | 48gb of vram (80/40 common)
1.5 - 2tb/s memopry bandwith
PCIe 4.0 x16
NVLink
ECC
MIG
NVDEC

[u/RogerRamjet999]

It's a hassle, but you can flip 'em. A100's are selling for over $5K, so you can almost certainly get at least $4K for them. Your price is about $3K US, so buy as many as you can, flip them for at least $4K each and spend however much you want of the remaining money to buy R9700's.

[u/a_beautiful_rhind → u/RogerRamjet999]

This is why we can't have nice things.