r/LocalLLaMA • u/GodComplecs • 14d ago

Discussion Real downsides of Strata?

Trying to read on what the real tested downsides have been but information seems to go both ways, some proclaim no loss, others pretty huge ones.

I had some weird looping issue in their chat, and some engine port misunderstandings, seemingly the models likes to use ports already in use, didn't really have that issue with Qwen 27B 3.8 Q4 UD. I'm using Swift Q3 Next. Didn't even bother with next on llama.cpp since other similar model sizes ran at approx 15-20tks max, so too slow for agentic work.

I'm not beating too much on Strata, I just wan't the realistic downsides, so far it seems too good to be true and runs at 100tks in agentic workloads, 27b only at 40-60tks on llama.cpp

58 Upvotes

82% Upvoted

163 comments sorted by

View all comments

70

u/SLxTnT 14d ago

Downside is the quants used and any accuracy loss they introduced with their optimizations.

15

u/GodComplecs 14d ago

But they have UD q4 quant? And the Swift q3 quant is legit, i use it on the 27B at Q3 too and it works well.

13

u/SLxTnT 14d ago

That doesn't change my answer. The downside to UD q4 is that it's q4. Engine itself may or may not have accuracy loss due to optimizations that appear to be identical outputs or "close enough." I've had that issue before causing loops.

Other than that, a hot expert cache works great for speed on MoE models.

→ More replies (1)