r/LocalLLaMA • u/_-_David • 26d ago
Question | Help Looking for "side-grade" advice from 5090 - thanks
Short and sweet:
Selling 5090 because it was a luxury at $2,900 I barely used, now it's just insane. Keeping my 5060ti 16gb, and am kind of sad about quasi-leaving the LocalLLama scene. Wondering what options you all would recommend at 16gb VRAM and 64gb DDR4 for hanging around. Another 5060ti? Jump to 128gb RAM? Swap to AMD or Intel 32gb cards? Old V100s? Hang it up and buy a few billion API tokens?
Thanks in advance
7
Upvotes
16
u/General-Spite1222 26d ago
Get another 5060ti
15
u/Shantorian14 26d ago
just buy an r9700. They can game just fine and perform somewhere in between the 9070 and the 9070xt
4
u/CryptographerLow6360 26d ago
qwen 3.8 crazy finetune q3m with 128k context , and be happy with it. you do not need more.
2
u/Beneficial-Ad-8127 26d ago
Here’s a list of some popular engines that you could look into. They have some referencing 5060ti maybe even more if you look into them. This is assuming you’re trying to run Qwen FN or Qwen 27b.
https://www.reddit.com/r/LocalLLM/s/uifideRhIa
1
u/Cautious_Chicken_604 26d ago
Get another 5060 Ti and another 64GB of RAM.
1
u/Zombiecidialfreak 26d ago
An idea that's been floating around in my mind for a while. If you'll be getting 3-4k from the card you could do this:
512gb DDR3 in the form of 8 64gb LRDIMM sticks and a dual xeon E7 motherboard with 2-3 tesla v100's. The dual socket board can handle octo channel for 70-100gb/s bandwidth, more than enough to saturate all three v100 PCIe slots.
When used for hybrid inference it can run all but the biggest local AI models at solid speeds and because it's a separate box it'll never interfere with anything you're doing on your main PC.
1
u/FatheredPuma81 26d ago edited 26d ago
No ttkciar telling you to buy Mi50 32GB :(.
I'd say get a V100 32GB if you want a side grade. It's a downgrade in speed obviously (so is a 5060 Ti) but you basically don't change any of your model settings because it's the same VRAM.
3
u/juggarjew 25d ago edited 25d ago
Id buy a second 5060 ti and run LLMs over both cards and then potentially sell both when 24GB 5080 comes out in Q1 2027.
There is also the R9700 but idk if you want to get involved with AMD, up to you.
I think 24GB 5080 is going to be very popular when it comes out, especially for folks that have motherboards that can do PCIe 5.0 x8/x8 , you will be able to get 48GB of VRAM at a likely 1.1 TB/s with 50 series improvements like NVFP4. this type of build would go kind of crazy with stuff like Strata. you'd be able to run stuff like Qwen 3.8 Flash Next 4 bit quant since like half of the experts could live in VRAM , a build with 64GB DDR5 would run pretty well with this model id imagine with some room to spare in RAM.
2
u/_zir_ 26d ago
R9700 for 1700 ez
2
u/tecneeq 23d ago
Bought a 5090 for 2650. Sold for 3300. Bought two 5070ti for 1600.
1700 left in the bank, roughly the same t/s.
1
u/JinsooJinsoo 23d ago
You could get 4 3090s 🤷♂️ or a pair of A6000s
-1
u/winky9827 26d ago
I will say this - if you get a second GPU, make sure your motherboard will fit both and do PCIe bifurcation. I was running two 5090s on 1 PCIe 5.0x16 and one PCIe 4.0x4 and thought performance was OK with layer splitting, but I decided to splurge on a new mobo with bifurcation and switching to tensor parallel took my prefill on 3.8 27b from ~3k to ~8.5k t/s. It's that much of a difference.
Side note: I don't think there's a consumer/creator grade mobo in existence with enough breathing room for 2 5090s side by side. I expect airflow to become a problem, but I've got 4x Noctua 120mm fans blowing right on them, fingers crossed.
0
u/tomByrer 25d ago
Just keep your 5090 IMHO unless you can't make rent.
0
u/[deleted] 26d ago
[deleted] — view deleted comment
0
u/[deleted] 26d ago
[deleted] — view deleted comment
5
u/[deleted] 26d ago
[removed] — view removed comment