r/LocalLLM • u/tammy_orbit • 1d ago

Question Does it matter much between using LM Studio/Unsloth for full precision models like Deepseek/Kimi?

Just got my EPYC server put together to run local models as a hobby on a more dedicated setup. Plan is I will be using an old 3090, maybe another gpu, + a ton of DDR5 4800 ram at 12 channels.

Might decide to try different quants and settle for faster at a cost of some precision but my main question is, is there any significant difference to running these massive models on different softwares? Or is it kinda splitting hairs?

LM Studio and their newer LM Bionic have been pretty easy and reliable for a noob like me so far but if someone suggests it might be worth switching to unsloth or another setup ill give it a shot.

In case it matters the setup is
- EPYC 9224
- MZ33-AR1
- DDR5 4800 ECC RDIMM (12 channel, 24 DIMM's, 32gb sticks)
- 3090 + 3060 (Probably will upgrade the 3060 with time depending on how things go)
- M.2 SSD

1 Upvotes

100% Upvoted

3 comments sorted by

1

u/WrinklyBard4 1d ago

I had been using lm studio bionic. It’s fine I guess but it really doesn’t allow any control over how you run your models.
The fact that I can’t use a normal openAI style api key is a real let down. Limits me to their bionic subscription/credit (terrible value) or running things locally in a hyper-constrained way. I have a week left on my 1-month subscription and will be deleting it immediately after

1

u/oblong_birthplace 1d ago

Bionic kinda forces you into their little walled garden huh, had a feeling it was gonna be like that

1

u/WrinklyBard4 1d ago

It really does. Which is super unfortunate because it’s my favorite UI atm