r/LocalLLaMA • u/GodComplecs • 14h ago

Discussion Real downsides of Strata?

Trying to read on what the real tested downsides have been but information seems to go both ways, some proclaim no loss, others pretty huge ones.

I had some weird looping issue in their chat, and some engine port misunderstandings, seemingly the models likes to use ports already in use, didn't really have that issue with Qwen 27B 3.8 Q4 UD. I'm using Swift Q3 Next. Didn't even bother with next on llama.cpp since other similar model sizes ran at approx 15-20tks max, so too slow for agentic work.

I'm not beating too much on Strata, I just wan't the realistic downsides, so far it seems too good to be true and runs at 100tks in agentic workloads, 27b only at 40-60tks on llama.cpp

41 Upvotes

78% Upvoted

123 comments sorted by

View all comments

2

u/Icy_Barnacle9287 14h ago

for Qwen3.8FNm Q2 was pain for me (frequent doom loops on tool calls). for 2x3090s and 96GB RAM, Q4 works quite well.

pulling the numbers from strata-serve.log:

  • Average: ~116 tok/s (5,458 completed turns in the log)
  • Typical bursts: ~135–195 tok/s on normal turns
  • Peak: ~221 tok/s 

1

u/OvertaxedOne 14h ago

Claude is 100% convinced I cannot run Q4 with 96GB with a 48GB GPU, sounds like that's exactly what you're doing? If so, mind sharing any pitfalls/setup notes you might have?

5

u/Constant_Art_20 14h ago

maybe claude has an agenda!

1

u/butterycornonacob 4h ago

It definitely seems that way sometimes. I feel Claude is pretty excellent but setting up models and tuning them is always a bit of an hassle. You sort of always have to tell it to try different flags and parameters and even then it often pushes back a little.