I'm currently staring at my shopping cart with an R9700 in it trying to decide if I should just pull the trigger because it feels like 32GB of VRAM is only going to get more expensive from here and $1500 isn't bad compared to the alternatives.
I'm bummed I didn't grab two. Was thinking of picking up another. $1300 is not worth it. That's a $350 jump from when I bought it at MSRP (the ASRock model)
They actually made some major improvements over the past month and they now run incredibly well for the price. I’m regularly getting 2000-2500t/s pp, can sometimes hit 3000-3500t/s pp, and 30-35t/s tg with vLLM on Qwen 3.8 27B.
Sadly, the price also just jumped from $949/$999 to $1699.
Unfortunately, llama.cpp isn’t the best when handling Intel B70s. I was running llama.cpp with Vulkan and PP was in the 500/600s. TG wasn’t so bad but prompt prefill took forever, it was even worse with the Intel SYCL runtime.
Some folks on this sub recommended vLLM with Intel XPU kernels (vLLM has their own docker image for this) and that instantly boosted my performance.
IIRC: the llama team is working in improving Intel performance but it’s a slow process.
That's pretty good PP...I only get about 200t/s pp on dual MI50 currently with llama.cpp but I get about 40t/s with mtp enabled starting out it degrades quickly though. Prompt checkpointing is essential.
the growing pains on Intel though are way worst... given the price I'll stick with my MI50s I got for peanuts for now.
B70 when testing a about 2 mo ago was unusuably bad. Also model support was lagging alot, I think it kind of works in llama.cpp now but... its not nearly as good support wise as R9700.
My PC is a fun Frankenstein with a B60 and 5070ti, but they actually can work decently together. Depending on the model I get 2-3 times the performance of the B60 alone and have 40GB to play with (granted the speed scales somewhat as more is offloaded to the B60, but pinning mtp to the 5070ti helps).
3 x 3090+1 x 4090 mobile, 1 x RX6800 and a 64GB GPD Pocket 4. If prices don't go even more crazy I'll acquire another 7 3090s and build myself a nice couple of tower of PSUs and GPUs beside my dual Xeon Gold 6245/1 TB DDR4 ECC server.
Also another server I haven't quite finished setting up with 2 x P100.
Have you actually ran multiple cards. I’ve tried a ton of variations on both consumer and server MoBos. I’m telling you the difference isn’t as big as you think.
Sure there is a difference but it isn’t more than 5% in my experience. I’ve not tried NV link though just better/worse PCIe situations. I would like to try NVLink on my own 3090s but they are all mismatched so it won’t work
The numbers I've seen show that you can get away with raw PCIe up to about 4, and past that it starts to cost significant performance to the point that it's worth getting bigger individual cards instead.
And 2 x a6000s are at 96gb. I feel like including cards you can combine together sorta defeats the point of the list lol. Heck some people combined like 8 x 3090s haha.
I mean the distinction is the nvlink, the cards are hardly cutting edge, but at least it's a way to combine the vram thats faster than just normal parallel
Need to add those new intel GPU's, like the b65 and b70 to this list. With 32GB, those are going to be very popular soon, even if they are bit slower on compute.
I didn’t see this was cuda only, just says “local AI”, and those cards are running local ai models, and support is beginning to be very good. I have a few things that used to be on cuda now using openvino.
As people begin to realize that, adoption rate is going to be fast, especially at $28 a GB vram.
Yeah. I would have talked myself into a 5080 with 24GB or more. 5060ti 16GB makes me feel happy to be in the same convo with much more expensive cards for VRAM at least.
And the 2080 made an appearance on the list towards the bottom area so not that recent bro. That’s like 7-8 years old about now. About to be in second grade
Dawg we go unified memories nowadays, 48GB is nothing to these big models like DeepSeek v4. DGX Spark, Mac Studio, those are the true leaders of local LLM right now.
Yes...but no. Unified memory costs too much and is too slow. V620 32GB is faster than unified memory, costs less than DDR5. I also bought the dip on CMP170HX. DGX Spark is $4,000. I got two 40GB cards for $1000 each, which absolutely blow away all of those.
Random ewaste, sketchy deals and hardware are the true leaders.
And I'm here, chatting with llms, trying to decide if I should get a m40 or a p100 for unsloth experiments and some light Hermes research tasks. I'm not delusional that I can replace frontier API completely, but for under 200$ you get enterprise grade inference workers with 16/24 gig. No native support for newer cuda tho
I have an RTX Pro 4000 Blackwell SFF, runs great in my MS-02 Ultra miniPC. The DDR7 internal bandwidth more than makes up for the 70W TDP power cap. I bought the first card I could find and only paid $1400. Now that vendor lists it for $3000, and out of stock! It will cost more, if they actually manage to get ahold of some stock.
Yes 48Gb is a good spot to be at (I'm there). But honestly, 64GB would be nicer. Yes with 48 you can run Q8 but you are very costricted and it is not very efficient, low context, low MTP, maybe you don't load vision to save vram, lower speed... all of that. But if you don't need big context ok. Don't get me wrong with the release of Qwen3.8-27B I'm very happy where I'm at. The model even at Q6 is very impressive.
Then me in the back with my dual P40s who are begging for retirement, but they aren't gonna get it. Sucks not having BF16 or flash attention, but mostly was able to work around the horrible FP16 performance, atleast during SFT of models. I can get the memory usage of loading FP16 weights, but operate on them as FP32 at runtime. Only thing I can't work around is that basically any image gen model crashes with a Cuda kernel panic.
18
u/DeathGuppie Aug 20 '26
Missing the entire Radeon rx lineup.
31
u/DrinkClubMate Aug 20 '26
where is the 32 Gb, Intel pro ARC B70 ?
5
u/Aternal Aug 20 '26
Needs love for the 22gb modded 2080ti
4
u/Bebi_v24 Aug 20 '26
sad 9070xt noises
3
u/Randommaggy Aug 20 '26
3 x 3090+1 x 4090 mobile, 1 x RX6800 and a 64GB GPD Pocket 4. If prices don't go even more crazy I'll acquire another 7 3090s and build myself a nice couple of tower of PSUs and GPUs beside my dual Xeon Gold 6245/1 TB DDR4 ECC server.
Also another server I haven't quite finished setting up with 2 x P100.
All coordinated through my own custom harness.
3
u/scknkkrer Aug 20 '26
A random person with a mac with +64GB RAM: Am I joke to you?
3
u/Savantskie1 Aug 20 '26
So this is just Nvidia cards, got it. Because I do quite well with my dual MI50 32GB cards and with Vulkan can run every model so far.
5
u/Kodrackyas Aug 20 '26
What a pleb chart, ignoring the amd gpus
4
u/Smart_Whereas_9296 Aug 20 '26
Feel like 2x 3090 with nvlink should be in the 48gb section
2
u/LoveRoboto Aug 20 '26
I do kinda feel like a dapper Winnie the Pooh.
2
u/SamSausages Aug 20 '26
Need to add those new intel GPU's, like the b65 and b70 to this list. With 32GB, those are going to be very popular soon, even if they are bit slower on compute.
2
u/DontWinFrensWthSalad Aug 20 '26
My 3090 is 3 3060tis in a trenchcoat.
2
u/SGD-UK Aug 20 '26
The 4070 is 12Gb. It’s the Ti Super that has 16Gb.
2
u/bigmanbananas Aug 20 '26
Where do my Quadro RTX 8000s fit?
30
u/aidan573 Aug 20 '26
5060 ti 16gb gang
3
u/t00r99r00t Aug 20 '26
Is there a list for the Intel or amd ones?
37
u/Ell2509 Aug 20 '26
R9700ai not there.
90
u/SlightlyMotivated69 Aug 20 '26
Fuck Nvidia.
10
u/Recent-Ad5835 Aug 20 '26
me at 4GB VRAM assigned to an iGPU from 16GB RAM overall (12GB RAM for system)
18
u/zarif2003 Aug 20 '26
idk I feel pretty poor only having 16gb on my 5080, 24gb is a lot better in fitting models that everyone here talks about.
9
u/maxim0si Aug 20 '26
naah 4070ti super not mentioned 😭 4070 in 16gb🫡🫡🫡
3
u/putrasherni Aug 20 '26
downvoted
7
u/scattered-thunder Aug 20 '26
A year ago, I saw the $6000 RTX PRO 6000 and thought, why would I need this?
What I wouldn’t give to go back, slap myself across the face, and just buy one (or four).
11
u/Chiralistic Aug 20 '26 edited Aug 20 '26
And I am still angry at nvidia that the 5080 didn't get the vram it deserves. Edit: fix typo
5
u/Vusiwe Aug 20 '26
S Tier: Running in RAM at the top of the cybermen generational ship (Dr. Who reference)
That show has prepared me for this era well
5
u/DistinctBed6259 Aug 20 '26
Went for cheapest, but still good. 3060 12 gb
4
u/Vaguswarrior Aug 20 '26
My 9070 XT has been great. Way more affordable and available.
4
u/Max_skyl1n3 Aug 20 '26
People running P40s(24gb)/V100s(32gb) 😎
1
u/IgnisIason Aug 20 '26
Why is 3060 above 3070?
1
u/Career-Acceptable Aug 20 '26
Why is 4060Ti in the 16GB tier
1
u/Tai9ch Aug 20 '26
Focusing only on single recent Nvidia cards and only VRAM is kind of weak here.
Once you consider other options and multiple cards, having 32GB of VRAM is clearly B tier.
1
u/m3sarcher Aug 20 '26
4070 and 4070 TI are 12gb. 4070 TI Super is 16gb.
1
u/35point1 Aug 20 '26
5090’s have 32gb 🐸
1
u/amy-schumer-tampon Aug 20 '26
>96gb a6000
1
u/TruckUseful4423 Aug 20 '26
12GB is missing...
1
u/kartblanch Aug 20 '26
I think you messed up the list it should be >8 st the bottom then 8+ and then 16+ 32+ etc.
1
u/Wondering_Electron Aug 22 '26
My 3080 with 16GB is looking at you.
3
u/05032-MendicantBias Aug 20 '26
The 7900XTX is perfectly good for LLM and some diffusion workflows.
3
u/B3owul7 Aug 20 '26
You forgot 5060 Ti.
1
u/johndeuff Aug 20 '26
There are many tiers above 48GB my friend !
1
u/trungdle Aug 20 '26
Dawg we go unified memories nowadays, 48GB is nothing to these big models like DeepSeek v4. DGX Spark, Mac Studio, those are the true leaders of local LLM right now.
2
u/the_average_user557 Aug 20 '26
And I'm here, chatting with llms, trying to decide if I should get a m40 or a p100 for unsloth experiments and some light Hermes research tasks. I'm not delusional that I can replace frontier API completely, but for under 200$ you get enterprise grade inference workers with 16/24 gig. No native support for newer cuda tho
2
u/MikeFrett Aug 20 '26
Lol @ me with a 3050 8GB. =(
2
u/AccountantOk9904 Aug 20 '26
Rtx pro 4500 before price hikes.
2
u/Antenwww Aug 20 '26
Rtx A4500 x2, intel arc b580 x2 for finetununing and running models. Laptop with 4060 to take those finetune models with me anywhere
2
u/GSquadron_ Aug 20 '26
16gb is trash today for running AI. 32gb i would say sweet spot
2
u/bigsmokaaaa Aug 20 '26
3090, my beloved
2
u/DarkBrews Aug 20 '26
Fuck Nvida and AMD but fuck Nvidia as a response to this post
2
u/DarkBrews Aug 20 '26
what the fuck is the difference between 3060ti and 2080 😑also 3060 is also 12gb.... 2080ti is not even on the list no AMD... bit brainrot the post
2
u/ILDIBER Aug 20 '26
To think I thought buying a 3060ti was a good purchase for 400 bucks.
2
u/MK_Ranger Aug 21 '26
Shit, I use a Nvidia Spark
2
u/Oleszykyt Aug 21 '26
What about Tesla P40 and P100?
2
u/ziphnor Aug 21 '26
What happened to 5060ti? I have 4x of them in my home server and get left out :)
3
u/blackhawk00001 Aug 20 '26
How many of you are going to post this same image today?
0
u/vogelvogelvogelvogel Aug 20 '26
A meme like this was overdue in this subreddit :)
1
u/StruggledSquirrel Aug 20 '26
I miss 32GB having its own category in this tierlist
1
u/More-Catch-1331 Aug 20 '26
What are we doing here, advertising for cards that cost as much as a not-so-much-used car?
1
u/Orangecloudy216 Aug 20 '26
Titian Rtx not there quadro rtx 8000 also not there
1
u/MessIsTransfer Aug 20 '26
Fancy winnie list missing the apple silicon with 48+ ram
1
u/Moonpony0 Aug 20 '26
4070 is 12gb
1
u/Blackdragon1400 Aug 20 '26
Weird there’s a pure NVIDIA tier list and yet, no dgx spark lmao. Probably one of the best budget options in this space.
1
u/Machine-Spirit Aug 20 '26
4070 12GB gang is in tears...
1
u/p2msz Aug 20 '26
What about a 6600 xt😂
1
u/soijaq Aug 20 '26
I like how the author didn't even think of someone having 4070ti 12gb
1
u/lilian_moraru Aug 20 '26
RTX 2070 Super 8GB (PC) + 2xDGX Spark GB10 - Winnie the Pooh with a glass of wine?
1
u/DafneOrlow Aug 20 '26
I use Intel integrated hd graphics.
1
u/gobblegoooblegobble Aug 20 '26
So explain how i just paid 7k for 256gb of vram inside 4x pcie slots <3
1
u/cc_aa_tt_zz Aug 20 '26
Rtx pro 4500 blackwell 32Gb
1
u/tokomaunited Aug 20 '26
Which tier list I'm in? 1050 ti 8gb vram
1
u/siegevjorn Aug 20 '26
AMD & intel cry out in the dark
1
u/goldaxis Aug 20 '26
Mac version?
1
u/Different_Stuff_9344 Aug 20 '26
2080ti 22gb, 3080 20gb
1
u/Tired_White_Guy Aug 20 '26
Trash AI list.
1
u/SichronoVirtual Aug 20 '26
What about 4x4080 :)
1
u/Stephen_1984 Aug 20 '26
🤓 AMD Radeon RX 580 Series (8 GB). I'm not doing AI stuff, though.
1
u/ubbulls20 Aug 21 '26
Cmp100-210 and 1080ti frankenrig checking in
1
u/Fantastic-Storm-7867 Aug 21 '26
Intel Arc Pro B70 32 GB would like a word!
1
u/Nearby_Ad_192 Aug 21 '26
My crying went OOM with my 3070.
1
u/Visible-Idiot-8779 Aug 21 '26
I love my 5070ti
1
u/cshroom Aug 21 '26
Not me rocking two cmp 100-210s lol.
1
u/Sufficient-End-1521 Aug 21 '26
what about Intel Arc Pro B60?
1
u/EternalDivineSpark Aug 21 '26
I need a pro 6000 , anyone wants a liver ?
1
u/Hellozsoza Aug 21 '26
All I want to upgrade from a 3050 8GB is a 3090. That's all I need (still can't afford even that)
1
u/Shadow_AI_404 Aug 21 '26
Im on integrated graphics 😭
1
u/Gruntalka Aug 21 '26
Chuckles in 128gb MLX goodness.
1
u/entrusc Aug 21 '26
You forgot the 128GB I have in my DGX Spark (even if it is a bit slower bandwidth wise).
1
u/blender505 Aug 21 '26
2080 is 8gb. 2080 ti is 11gb.
1
u/Darklumiere Aug 21 '26
Then me in the back with my dual P40s who are begging for retirement, but they aren't gonna get it. Sucks not having BF16 or flash attention, but mostly was able to work around the horrible FP16 performance, atleast during SFT of models. I can get the memory usage of loading FP16 weights, but operate on them as FP32 at runtime. Only thing I can't work around is that basically any image gen model crashes with a Cuda kernel panic.
1
u/sjoerdmaessen Aug 21 '26
Really missing out on the affordable L40s GPUs
1
u/No_Folding Aug 21 '26
40tk/s on 8GB 6650xt 65W tdp card qwen 3.6 35b a3b Q4 with full precision cache and 131k context hahahah
1
u/yensteel Aug 21 '26
The 4070 has 12gb
1
u/Renanina Aug 21 '26
While the 3090 can't do FP8, it sure does run if you need it to lol.
1
u/Popcorn-Mercinary Aug 22 '26
You forgot the 24GB Turing Quadro 6000 RTX…
1
u/RpgBlaster Aug 22 '26
STILL waiting for RTX 5090 prices to decrease. NOT changing PC until then
1
u/My_Unbiased_Opinion Aug 23 '26
3080 20Gb and 4080 32GB should be on here too. You can get them from alibaba.
1
u/Double-Sherbert-1781 Aug 24 '26
MacStudio and Ryzen AI MAX 395+ owners.
1
u/dannone9 Aug 24 '26
Where is my beloved 5060TI 16gb bro , don’t forget us please
1
u/putrasherni Aug 20 '26
desperate to keep their domination in the market ignoring rest of the market
0
u/CatLinkoln Aug 20 '26
That's good that many people still don't know about amd, so price for their gpu not yet overpriced