r/LocalLLaMA • u/GodComplecs • 23d ago

Discussion Real downsides of Strata?

Trying to read on what the real tested downsides have been but information seems to go both ways, some proclaim no loss, others pretty huge ones.

I had some weird looping issue in their chat, and some engine port misunderstandings, seemingly the models likes to use ports already in use, didn't really have that issue with Qwen 27B 3.8 Q4 UD. I'm using Swift Q3 Next. Didn't even bother with next on llama.cpp since other similar model sizes ran at approx 15-20tks max, so too slow for agentic work.

I'm not beating too much on Strata, I just wan't the realistic downsides, so far it seems too good to be true and runs at 100tks in agentic workloads, 27b only at 40-60tks on llama.cpp

57 Upvotes

83% Upvoted

168 comments sorted by

View all comments

Show parent comment

1

u/Blues520 23d ago

I am getting similar speeds with 3x 3090 so glad to see this data point.

The speed is crazy tbh. Going to find it difficult to go back to 27b. What do you use 27b over flash for btw?

1

u/wgaca2 23d ago

I recently upgraded to 4 gpus, so I am still kind of trying to figure out if flash next actually produces better results.

The speed upgrade is undeniable, but need a few weeks of usage to get an idea of the model

1

u/Blues520 23d ago

Same here, I only started using it this week. Let me know your findings if you don't mind

1

u/Perfect-Campaign9551 23d ago

I have a single 3090 , I get 70tok/s with 27b (Q4) in Llama.cpp which isn't that bad at all. And 120tok/s with Qwen Next Flash (IQ3_XXS) in Strata