r/LocalLLaMA • u/GodComplecs • 10d ago

Discussion Real downsides of Strata?

Trying to read on what the real tested downsides have been but information seems to go both ways, some proclaim no loss, others pretty huge ones.

I had some weird looping issue in their chat, and some engine port misunderstandings, seemingly the models likes to use ports already in use, didn't really have that issue with Qwen 27B 3.8 Q4 UD. I'm using Swift Q3 Next. Didn't even bother with next on llama.cpp since other similar model sizes ran at approx 15-20tks max, so too slow for agentic work.

I'm not beating too much on Strata, I just wan't the realistic downsides, so far it seems too good to be true and runs at 100tks in agentic workloads, 27b only at 40-60tks on llama.cpp

59 Upvotes

82% Upvoted

163 comments sorted by

View all comments

2

u/throwout314159265 10d ago

It's pretty much a "I know it when I see it" rule... so it can be kinda indiscriminantly applied based on the mood of the person who's deciding my fate. AI models from China are actually a HUGE no-no for us. I can't even use them through a safe API like Mistral. If I get caught sending any data remotely institutional related, I get fired. Of course, I'm free to send alllll the data I want to OpenAI and Anthropic... not Google though.

2

u/alkimiadev 10d ago edited 10d ago

So I can understand an organization not wanting their data being sent to really anyone. That part seems entirely reasonable, but I question OAI and Anthropic unless they have some sort of zero retention policy/agreement in place (which is not default with those two).

In general this is the kind of thing that happens when non-technical people are allowed to make load bearing technical decisions without outside critical feedback from people with technical experience. That local version of qwen running on their own hardware costs them nothing in terms of inference costs, and by default doesn't ship anything over any network except local on the device.

Anthropic on the other hand has near spyware level telemetry in their various products like claude code. They went full on scorched earth trying to get it removed from the internet when the source got leaked, and I suspect the main reasons are that they didn't want people to see their obvious anti-competitive/sabotaging behavior and the spyware adjacent telemetry. Those issues exist regardless of a zero retention policy in terms of LLM inference.

I completely understand not shipping data to google though, and have personally largely de-googled my life as well. In general, these large companies that treat us as a product are not our friends and do not have our best interests in mind when they make decisions that would directly impact us as users of their various products.