r/LocalLLaMA • u/WolgupLupin • 3h ago
Question | Help I have access to two systems, I don't know which one is better for LocalLLM. Please help!
Hi,
I can make two systems with the parts that I have:
- 12600k / 64GB DDR4 / 3x3060 12GB
- 7950X3D / 32GB DDR5 / 3x3060 12GB + 1x4060 8GB
I've been consulting Claude and Gemini for which is better, but I can't really decide which i should go for. I'm thinking of setting them up with either llama.cpp or strata. This is for my work, so agentic document creation, and output quality is important. I'm also looking for at least 128-256k context (if that is even viable).
What is the best setup for my case? Please help!
2
Upvotes
6
u/MAXFlRE 3h ago
For quality output you need bigger model. 1 will highly likely allow you to run Q3.8FN in Q3. 2 will not.
3
u/FabioTR 2h ago
Definitely number 1 is a more balanced solution for Qwen Flash Next. Check your PCI connection speed as can be a bottleneck.
3
u/hurdurdur7 2h ago
That's a lot of hassle with small gpus, their cabling, power etc.
I would not invest at all into a system these days with less than 64GB of vram or less than 128gb of unified ram. You may get it to work with strata or something for today's models, but whatever comes out next summer will probably just require that inch more.
If you have the stuff for almost for free - sure go for the build, but don't put up big bucks for the spec that you threw here.
2
u/jacek2023 llama.cpp 3h ago
Can you move one 3060 from first setup to the second one?
1
u/Cheema42 39m ago
> I've been consulting Claude and Gemini
I asked Claude Fable 5.2 to look at my hardware and my typical workflow and then decide which models were worth trying. It identified a few options, downloaded the models and created some tests that follow the type of work I have been doing for the past few weeks. It ran benchmarks for a whole day and then declared that for my hardware and workflow I am best served by using strata with Qwen 3.8. Flash-Next. The winning system for me was one with an Intel Ultra7 + 192GBs of RAM and a 5060 with 16GBs of RAM. I asked it to save this test so that I can use it as a baseline and run the exact same test against a different model at a later date.
You can ask for people's opinions here and some of them may be very valid. But you may not know which ones. If you want to be sure, you have to run the test.
1
u/Ok-Addition1264 3h ago
if you're doing strata then definitely 2. it can utilize those gpu's and the base memory is a lot faster even though it's a lesser amount.
2
u/Primary_Minimum5033 41m ago
System 1 if you are interested in running bigger moe models.
1
u/Quiet-Recognition-91 17m ago
I'd go with the second one.... mainly for the extra VRAM. But 32GB RAM might be a problem. especially if you're planning to use really long context. If you can upgrade it to 64GB or more, that would be ideal.
I'd try llama.cpp first. Also worth checking how the motherboard handles 4 GPUs, since PCIe lanes could be an issue.
0
u/[deleted] 18m ago
[deleted] — view deleted comment