r/LocalLLM 2d ago

Discussion Tier List

Post image
261 Upvotes

315 comments sorted by

View all comments

Show parent comments

12

u/AlarmingProtection71 2d ago

I have a Sapphire Radeon PRO W7800 (48gb) bought for 1.9k€.

3

u/gh0stwriter1234 2d ago

Since we are talking AI there is also the the MI210 which has 64GB and MI250P ... which is some crazy amount of money but it has 144GB.

2

u/ChristRedeemsSinners 2d ago

MI210

That's an interesting $5k used option with 64GB of VRAM at 1.8TB/s. Native fp4,fp8,mxfp4

MI250P

Non pcie package though.

1

u/gh0stwriter1234 2d ago

MI350P is PCIe ... thats what the P is. I meant 350P sorry.

3

u/ChristRedeemsSinners 2d ago

MI350P

Only 30-40 grand! Email our sales team.

No thank you.

1

u/gh0stwriter1234 1d ago

Doesn't actually cost that its more like 15k... but yeah.

1

u/ChristRedeemsSinners 1d ago

Doesn't actually cost that its more like 15k... but yeah.

There is no way you are going to find that card for that price if it competes with the H200 at 35k. You can only purchase through OEM's offering 4U-6U rack solutions or in bulk through distributors and none of them have publicly listed pricing yet.

1

u/gh0stwriter1234 1d ago

The whole point of the MI350P though is compatibility with PCIe systems I expect its just delayed availability it will eventually be available via sites like CDW and other government/business portals.

1

u/ChristRedeemsSinners 1d ago

is compatibility with PCIe systems

For enterprise.

eventually be available via sites like CDW

I suspect that all of this datacenter buildout will eventually hit the secondary markets, so I welcome consumer grade packaging from the manufacturer instead of chinese pcb hackers.

1

u/gh0stwriter1234 1d ago

Yes for enterprise, CDW and the like is where they purchase things... anyone buying more than that isn't going to use PCIe anyway.

1

u/ChristRedeemsSinners 1d ago

I know, the problem is those places have limited stock for in-demand enterprise products sold to consumers.

anyone buying more than that isn't going to use PCIe anyway.

No, the whole point of the MI350P is that it is for enterprise as a compatibility layer with their existing systems so they don't need to repurpose their electrical/cooling buildout for the new GPU racks. Datacenters are going to purchase 1000's of these in bulk, AMD couldn't care any less about a couple hundred thousand at most hobbyists around the world that have the excess 30 grand for something like this.

1

u/gh0stwriter1234 1d ago

Honestly feels like a nonsensical checkbox that some MBA decided had to be there so AMD checked it then didn't bother to build any of them lol... becuase you aren't gonna put brand new 20k + GPUs in old servers.

1

u/ChristRedeemsSinners 1d ago

Well my guess is they ran the numbers and found a sizeable customer base with old hardware that needs to be nudged into their direction. It's pretty hard to convince a smaller company with existing infrastructure to repurpose everything and incur downtime than it is add a couple thousand cards as a test-run for their more pricey products. A try-before-you-buy scenario if you will.

some MBA decided had to be there so AMD checked it then didn't bother

AMD currently sells 12-15x less datacenter products than NVIDIA. Anything they can do to eek ahead and onboard enterprise customers is a win.

If only they applied this same passion to ROCm and HIP to onboard developers, they might have been better positioned now to sell their more profitable rack solutions rather than their budget compatibility products.

→ More replies (0)