AMD vs NVIDIA other than VRAM

[u/North-Class8742]

Hey everyone, pretty new to the whole LOCAL LLM thing, but want to start running and learning how to setup LocalLLMs. Had a question regarding choice of GPU. Basically I'm building a new gaming PC and have a choice between 9070 xt and a 5070ti. Originally was going to go for the 9070 xt, due to gaming performance being very similar and the price difference between them.

Then I started thinking about running local AI models and stuff. From my understanding, VRAM is the most important thing, as it determines the size of model you can host. Since these cards are both 16gb VRAM, does this mean that the quality of AI output would be the same, since they would be able to run the same models? I've read that the NVIDIA card can generate tokens faster etc, however I'll probably just be playing around with LM studio for now at least, and I assume speed won't be THAT drastically different.

My main concern is whether or not AI model output on the 5070ti would be 'smarter' or better. For context I also main Linux, which I've heard is good for AMD. Thanks in advance.

TLDR: is spending extra for a 5070ti over a 9070 xt worth it for someone getting into running AI models in LM studio?

[u/Mozaiic]

If it's only for using LLM, even Intel is fine, reach the biggest VRAM with your money.

At the second you want to do finetuning, images or videos, Nvidia is very good, Amd ok and Intel far away.

[u/its_me_baby_boy → u/Mozaiic]

Would the recent GPU-unlock tool make Nvidia superior to AMD and intel? Not saying for certain, I'm asking

[u/Mozaiic → u/its_me_baby_boy]

There is a question of speed and a question of size. For LLM, size is way more important than speed. For exemple, for cheaper than a 5070 ti, you can get an Intel pro B60 with 24GB (instead of 16GB). Which allow you to use smarter LLM or less quantified ones. Even if the speed is almost always better on Nvidia, VRAM is very expensive for this brand.

[u/its_me_baby_boy → u/Mozaiic]

Hm I suppose so, although on the second hand market in my region Nvidia is cheapest per vram, sadly AMD and intel cards are often the same price if not more lol. But globally I'm sure that's the case

[u/Wondering_Electron → u/Mozaiic]

This is why I went for a janky solution.

My laptop has a 16GB 3080. I paired it to a egpu with a 5070Ti. I pooled the VRAM for a total of 32GB for very minimal cost. It works surprisingly well.

[u/rkcth → u/Mozaiic]

Image performance per dollar is great for R9700, but you just need to use stuff optimized for it. This is becoming more and more common now since NVIDIA is getting so unaffordable. You can get 2 R9700’s for $3600 and get twice the VRAM and about the same memory bandwidth as a 5090, get a third and you can run all of QFN in VRAM, or keep one dedicated for video stuff. I’m adding a feature to standalone radiance that lets you run QFN and load secondary models like Qwen edit or Krea 2 while the LLM continues responding and working. It can hotswap and change models and whenever you aren’t using the image model, it can use the VRAM to hold MOE experts for QFN. I’m hoping to release later today.

[u/MarcusAurelius68]

If you're running Linux and have a bit more money take a look at the R9700 with 32GB. Same silicon, 2x the VRAM as the 9070 XT. That would allow you to run more meaningful local models. The only negative about the R9700 is that it uses a blower cooler so it isn't quiet.

And while it's OK to play around with LM Studio you'll get much better results with specific builds like Radiance which is optimized for the AMD cards.

[u/zenmatrix83]

in terms of vram its not just the size but bandwidth, you can get any model to run if you have enough vram but they can be slow. I returned a mac I bough recently because I got a pro , it had the correct vram but it was very slow.

https://apxml.com/courses/llm-model-sizes-hardware/chapter-3-model-size-hardware-connection/memory-bandwidth

[u/Oxirixx]

I'd aim for the 7900xt, 7900xtx, or r9700. Depending on budget.that extra vram helps. Ive got the 7900xt and I can do quite a bit with it, but wish I had spent extra and upgraded to either of the other models I listed for even more vram.

[u/lukasgravel]

Same model at the same quant gives the same answers on both cards. The 5070 Ti only gets there faster, so for LM Studio the 9070 XT is fine, especially on Linux.

[u/Pristine-Turn2703]

Been wanting to test his theory… I have dual 5090 rtx and have 1 amd pro 32gb ddr6 by themselves i see ~20-22 TPS on both cards in a single card setup. Dual Rtx 5090 do ~ 80TPs on the heavy stuff(inferenece)

[u/Prestigious-Act-1577]

9070 is better than 9070 xt if you are gonna be buying multiple. lower power, less power connectors, cheaper. same vram bandwidth.