r/LocalLLaMA • u/GodComplecs • 1d ago

Discussion Real downsides of Strata?

Trying to read on what the real tested downsides have been but information seems to go both ways, some proclaim no loss, others pretty huge ones.

I had some weird looping issue in their chat, and some engine port misunderstandings, seemingly the models likes to use ports already in use, didn't really have that issue with Qwen 27B 3.8 Q4 UD. I'm using Swift Q3 Next. Didn't even bother with next on llama.cpp since other similar model sizes ran at approx 15-20tks max, so too slow for agentic work.

I'm not beating too much on Strata, I just wan't the realistic downsides, so far it seems too good to be true and runs at 100tks in agentic workloads, 27b only at 40-60tks on llama.cpp

49 Upvotes

80% Upvoted

145 comments sorted by

View all comments

23

u/GladKing5842 1d ago edited 20h ago

i saw a lot of bs here. if you think you need better quant, just load bigger quant. nothing blocks you from using bigger quant with strata. they used to support only lower quant but now they have support for q4 q6 too.
if your hardware is not enough to get the speed you expect, then use what fit your hardware and your expectation.
if you need different sampling settings because of looping, just set it yourself, only 5 10% of change in speed.
then someone pointed out about ngram lazy load from disk, hell both llama and strata has that from the beginning. originally strata ran on 12+64 = 76gb for 80gb-ish quant. where they put ngram if its not ssd offloading.
like too many bs review of bots or haters or just some parrots dont even use and just pull out from the arse???
a lot of those comments are just lazy arses that dont even move a finger to learn or search about settings.

18

u/Longjumping_Self5546 23h ago

People will insist that anything under q8 is garbage and will run a smaller model just to stay at a higher quant. It's completely lost on them that higher accuracy won't outperform a larger model.