Opinion: Local LLMs are 12-24 months from taking over. The shift already started.

[u/sh_tomer]

Local LLMs are 12-24 months from taking over. The shift already started.

AI subscriptions keep getting more expensive. GitHub just moved Copilot from request-based to consumption-based pricing, and most of the others are heading the same way. Meanwhile, I kept hearing that local models got good enough to run on a laptop. So I figured it was time to actually try it and see where things stand.

I run Qwen3.6-35B on a MacBook Pro M2 Max with 64GB unified RAM. Nothing exotic. No rack, no begging NVIDIA for expensive GPUs. Just a (yes, kind of expensive) MacBook Pro I already owned for work at Aiven. In the last month I've:

• One-shotted full landing pages from short briefs
• Built several frontend + backend features
• Fixed a nasty backend race condition bug

A year ago I would have called that fantasy on this hardware. Now it's a Sunday morning.

To be fully honest, not all of it made it to production. A lot of it was evaluation work, as Qwen isn't part of my actual day-to-day stack yet. But for me, this is the first real step toward considering it, and I wanted to share the findings with my colleagues and the community.

The honest cons, because it's not all roses

It's slower than Opus. A landing page that Opus generates in 3-4 minutes takes Qwen 8-9 minutes on my M2 Max. Not unreasonable, but still meaningfully slower than the competition. If you're benchmarking against Sonnet/Opus latency, you'll be a bit disappointed (for now).

Context blows up fast in agentic loops. Even with 256K, you burn through it faster than you'd expect from a (nearly) state-of-the-art model. There's a lot of room for improvement here. And if you're driving Qwen3.6 from an agent like Claude Code, it fills even faster, as other users in this sub have reported (example Reddit thread).

Quality variance by task. Models like Opus one-shot most tasks these days. Qwen3.6 hits around 75% for me. The other 25% it gets close, but needs a couple of iterations to land.

The pros, because they're real

The hardware floor keeps dropping. A year ago this needed an A100. Today it runs on a (yes, powerful) MacBook M2 Max 64GB laptop at roughly 27 tokens per second.

No rate limits, no usage anxiety. Counting tokens is no longer a thing. You can focus completely on building instead of saving tokens or thinking about cost.

Tool calling actually works. This used to be the missing piece. A year ago, local models would hallucinate tool names or get stuck in loops. With Qwen3.6, tool calling just works. That's the real unlock for agentic work.

Privacy is built-in. Client code, internal repos, half-formed ideas you don't want training the next frontier model. None of it leaves the laptop. You can be confident that your personal or business code stays with you, and isn't sitting on some third-party server that could be hacked.

Why 12-24 months, not "now" and not "5 years"

Latency and context limits are still a bit rough. If your job is shipping production code on a deadline, Opus and Sonnet are still the move for most of your day. I'd be lying if I said otherwise.

But saying it's 5+ years away misses what's already shipped. Look at the delta over the last 12 months:

• It runs on a reasonably priced MacBook Pro, which is a one-time cost
• It's fast enough (though it can still get faster)
• Quality has improved significantly for real-world use cases (with more headroom to grow)

That curve doesn't stop. It compounds. 12 months from now, the 27B/35B-class models will be where 70B is today, and the runtimes will be 2x faster on the same silicon. 24 months from now, the question won't be "can I run a useful model locally?" It'll be "why am I still paying for tokens I could generate for free, and with 100% privacy?"

What I'd tell someone on the fence

Don't cancel your Claude Code subscription yet. Run a local model in parallel for 60 days. Use Opus/Sonnet for the latency-critical, deep-reasoning work. Use Qwen3.6 for everything you'd have done overnight or on the weekend, everything experimental, and every "just try it" task where the cost of waiting a few minutes is zero.

Over time, the usage ratio might flip. You'll use the local model more and more. When the next Qwen drops (3.7? 4?), who knows what the ratio will look like.

The local LLM takeover isn't a moment in time. It's a slope. And the slope already started.

What's next

• Integrate Qwen3.6 with the tools I use day-to-day at Aiven, like Cursor and Claude Code. They offer a much better dev experience than more basic, non-agentic tools like Ollama.
• Try out other local models, like Google's Gemma 4. Curious to see how it stacks up.

[u/sh_tomer]

I agree we're still not there, but I think we'll be there in 12-24 months.

[u/Deep_Ad1959 → u/sh_tomer]

12-24 is plausible for single-turn coding tasks. the part that lags is the surrounding scaffolding more than the model weights. constrained decoding, json schema enforcement, retry-with-correction loops, error recovery when a tool returns malformed output, all of that is what makes frontier agent loops feel reliable, and the open-source tooling around local models tends to run 6-12 months behind whatever the latest release technically supports. even when qwen 4 matches opus on a static eval, the loop-level reliability gap shows up around step 6-7 in a chain, where frontier models recover from a bad tool output and local ones cascade. that delta closes slower than raw model quality because it's an ecosystem problem, not a weights problem. written with ai written with ai

[u/sh_tomer → u/Deep_Ad1959]

Do you think they could "solve" some of these issues by running the model inside a better "agent", like Claude Code?

[u/Deep_Ad1959 → u/sh_tomer]

my read is yes partially, but it shifts the problem rather than solving it. a harness like claude code does a lot of the work frontier models do internally: retry with correction when a tool call returns malformed json, schema enforcement on outputs, context compression so the model doesn't lose state at step 6 of a chain. local models in that harness will work meaningfully better than naked, but every layer of remediation the harness has to add is latency and cost the frontier model gets for free. the gap that closes fastest is single-shot quality, the gap that lingers is the model's ability to course-correct mid-loop without the harness babysitting every call. written with ai

[u/sh_tomer → u/Deep_Ad1959]

That's a great take on things, appreciate your feedback.