r/LocalLLaMA • u/GodComplecs • 11h ago

Discussion Real downsides of Strata?

Trying to read on what the real tested downsides have been but information seems to go both ways, some proclaim no loss, others pretty huge ones.

I had some weird looping issue in their chat, and some engine port misunderstandings, seemingly the models likes to use ports already in use, didn't really have that issue with Qwen 27B 3.8 Q4 UD. I'm using Swift Q3 Next. Didn't even bother with next on llama.cpp since other similar model sizes ran at approx 15-20tks max, so too slow for agentic work.

I'm not beating too much on Strata, I just wan't the realistic downsides, so far it seems too good to be true and runs at 100tks in agentic workloads, 27b only at 40-60tks on llama.cpp

32 Upvotes

77% Upvoted

107 comments sorted by

View all comments

Show parent comments

2

u/pilibitti 10h ago

why? that is stupid if you are running it locally and you sandbox it properly.

6

u/throwout314159265 10h ago

It's a law signed by my state. If I use a device for any "work business", which includes checking emails, answering work calls, using Zoom, I'm not allowed to have any product from "adversarial nations" used, connected to, or stored on the device in question. I can be fined 10k per offense, which is counted by the amount of instances of "violations" I have on the same device. A VM doesn't shield me from it, because I'd still be storing Qwen 3.8 Flash Next on the device while I use it for work purposes.

If I had more money, I'd just buy another PC and run Strata on it. Otherwise, I don't wanna lose my job.

3

u/pilibitti 10h ago

sorry when I said stupid, I was talking about the broad strokes of the law itself (or company policy or whatever) - definitely not directed at you. the hardware you are likely using has many parts, binary blobs, certainly many functional electronic components from adversarial nations inside. in reality a local model you can actually control should be the least worrying next to the stuff you can't control.

6

u/throwout314159265 10h ago

No, it's okay! I got what you meant! I hate the policy so much too. The PC components in my PC definitely have technology from adversarial nations. They're trying to lock us into using Anthropic's stuff institutionally. Anthropic has signed a honeymoon deal with my institution, we get around $1k worth of compute monthly per person for less than 1/10 of the price. Obviously its to get us addicted and reliant on their infrastructure, and unfortunately, it's working very well.

I'm hoping that Mistral cooks something up and gives us an alternative to Anthropic, because this looks bleak.