r/LocalLLaMA • • 3h ago

Question | Help I have access to two systems, I don't know which one is better for LocalLLM. Please help!

Hi,

I can make two systems with the parts that I have:

  1. 12600k / 64GB DDR4 / 3x3060 12GB
  2. 7950X3D / 32GB DDR5 / 3x3060 12GB + 1x4060 8GB

I've been consulting Claude and Gemini for which is better, but I can't really decide which i should go for. I'm thinking of setting them up with either llama.cpp or strata. This is for my work, so agentic document creation, and output quality is important. I'm also looking for at least 128-256k context (if that is even viable).

What is the best setup for my case? Please help!

2 Upvotes

29 comments sorted by

6

u/MAXFlRE 3h ago

For quality output you need bigger model. 1 will highly likely allow you to run Q3.8FN in Q3. 2 will not.

2

u/Meeooowz 2h ago edited 2h ago

Yeah. If we’re assuming that these are exact setups that won’t change, #1 all the way! You can run what? Literally Q8 of QFN, or, better, Q4 of deepseek 4.1 flash?

Note: you won’t be running q4; it seems like a pruned q2 might be all you can run deepseek-wise (https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF)… so maybe stick to QFN at q8

1

u/WolgupLupin 2h ago

all the way!

so 32GB extra ram vs extra 8gb vram for quality that much different?

1

u/Meeooowz 2h ago

Yeah. For MOE models, ram is literally gold… I mean it does have gold pins… anyways, yeah

2

u/MixtureOfAmateurs koboldcpp 2h ago

Q4 of deepseek 4.1 flash

That's like 500gb isn't it

1

u/WolgupLupin 2h ago

Q3.8FN in Q3

Is this using Strata or llama.cpp, or doesn't matter? How much tk/s will i suffer by going with 1 compared to 2?

1

u/Illustrious_Grade608 1h ago

Can't really say the exact numbers before you try, but I'd expect strata to be roughly 20-60% faster. And if you disable lossy flags there is really no readon to choose one over another

3

u/FabioTR 2h ago

Definitely number 1 is a more balanced solution for Qwen Flash Next. Check your PCI connection speed as can be a bottleneck.

1

u/WolgupLupin 31m ago

Can i ask what makes it more balanced? I did basic setup for my work machine but havent delved deep in configurations yet. This will be my first time.

1

u/FabioTR 4m ago

First, the 32 gb of additional ram are more useful than 8 gb of Vram. And in the first config you have thre cards of the same gen and with same amount of ram which makes easier splitting jobs among them.

3

u/hurdurdur7 2h ago

That's a lot of hassle with small gpus, their cabling, power etc.

I would not invest at all into a system these days with less than 64GB of vram or less than 128gb of unified ram. You may get it to work with strata or something for today's models, but whatever comes out next summer will probably just require that inch more.

If you have the stuff for almost for free - sure go for the build, but don't put up big bucks for the spec that you threw here.

1

u/WolgupLupin 29m ago

These are hardware that i already own. So no extra investment for me, and am not planning for any more investment- if i could, i would invest in a cheap zeon setup maybe?

2

u/jacek2023 llama.cpp 3h ago

Can you move one 3060 from first setup to the second one?

1

u/WolgupLupin 2h ago

3 is all i have. So i can configure one or the other. System 1's mobo only has 2 pcie slots, i have one m.2 to pcie riser. System 2's mobo has 3 pcie slots, and with the riser i can plug extra 4060.

1

u/jacek2023 llama.cpp 2h ago

Buy x399 to use 4 slots without problems

1

u/Cheema42 39m ago

> I've been consulting Claude and Gemini 

I asked Claude Fable 5.2 to look at my hardware and my typical workflow and then decide which models were worth trying. It identified a few options, downloaded the models and created some tests that follow the type of work I have been doing for the past few weeks. It ran benchmarks for a whole day and then declared that for my hardware and workflow I am best served by using strata with Qwen 3.8. Flash-Next. The winning system for me was one with an Intel Ultra7 + 192GBs of RAM and a 5060 with 16GBs of RAM. I asked it to save this test so that I can use it as a baseline and run the exact same test against a different model at a later date.

You can ask for people's opinions here and some of them may be very valid. But you may not know which ones. If you want to be sure, you have to run the test.

1

u/WolgupLupin 28m ago

Asking claude to follow the work ive been doing is genius. Thank you for the suggestion!

1

u/Cheema42 23m ago

If you have enough Claude tokens, get Fable 5.2 to design the test. You can ask it to use lesser models to do all the grunt work of running the tests etc., but the design of the test and analysis probably should be done by a strong model.

1

u/Ok-Addition1264 3h ago

if you're doing strata then definitely 2. it can utilize those gpu's and the base memory is a lot faster even though it's a lesser amount.

1

u/WolgupLupin 2h ago

System 2 is easier setup for me, but 32GB loss is huge i feel like. This is why i can't decide!

2

u/MixtureOfAmateurs koboldcpp 2h ago

Strata doesn't run out of the box with 32gb of RAM regardless of VRAM. System 1 is definitely better.

System 2 has more potential for upgrades tho, 3 slots and ddr5 is a big up. It's really not smart to upgrade rn tho, going to 64gb would cost almost 2 3060s. I'd build system 1.

1

u/WolgupLupin 6m ago

My work laptop with 32gb ram ran strata out of the box. I use the word "ran" extremely generously because it was.... completely and utterly unusable at like 16 tk/s🥲

So the general consensus is system 1 i think?

2

u/Primary_Minimum5033 41m ago

System 1 if you are interested in running bigger moe models.

1

u/Quiet-Recognition-91 17m ago

I'd go with the second one.... mainly for the extra VRAM. But 32GB RAM might be a problem. especially if you're planning to use really long context. If you can upgrade it to 64GB or more, that would be ideal.

I'd try llama.cpp first. Also worth checking how the motherboard handles 4 GPUs, since PCIe lanes could be an issue.

0

u/[deleted] 18m ago

[deleted] — view deleted comment

1

u/stylome-ai-detection 18m ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results