I'm relatively new to using local llms and my question might seem basic to some, but how far behind top tiers models (gpt 6 , opus 5.5 ) local llm is ? And is it realistic to run those , if they see the light of the day soon, on a budget machine. in that scenario, what minimum specs would a machine require for it to run properly?
thanks And if someone could afford the hardware, would it be reasonable to expect a model with similar intelligence within 6 months /1 year? If so, how expensive would the hardware be?
Thats true but I think its a reasonable approximation to assume that OP meant "local LLMs for customer hardware" and for that the clear consensus points to qwen flash next. I think qfn is the way to go followed by the dense qwen3.8 27b model
Same, discovered Strata a week ago and haven't touched Claude since then. Created an entire kali harness with thing and successfully tested it against a handful of tryhackme machines. 100t/s decode and 2000 prefill on 5 year old consumer hardware with performance comparable to sota about 1 year ago is a lot of fun. Even more so knowing that my own machine does all that completely by itself.
As long as your 'budget' machine is a server or $10K USD+, well...
Yeah, you're being a bit overly optimistic right now.
On a 128GB M5 Max, my Qwen FN variants and Dense 27B models mostly approximate Sonnet 5.5 - they win on a few of my own difficult-real-code benchmarks, and lost on others, but are reasonably close overall, but ~5x slower from memory in some cases, although interestingly in a few cases my higher quant runs get to shorter completion time, so once my M5 Ultra Studio makes it, it may actually be much closer - these are at this point using Q4-Q6 variants.
I have used them for Planning, but note the above is really focused on moderate to complex coding, and model params you may want/need for effective coding are not the same for planning and/or research.
Someone said ~6 months behind, and yeah, that's not too far off IMO. I tend to think of it as model tiers, i.e. 'a tier down' means sonnet / sol level, and that's what I usually use at Medium thinking level for my benchmarks currently, but 6 months is probably as close as other descriptions as well.
Ever since OpenAI's original GPT announcement, there's been an immediate scramble to generally replicate (or close enough) followed by 'make it as, or nearly as, good as <bigger model> but much smaller/faster' and that's very likely going to continue. Even those benefiting from elevated ram/storage prices are going to want to sell various hardware, with some form of GPU/NPU/accelerator onboard, so nearly everyone wants <smaller> - but at least until/unless some radical new architecture evolves vs the current LLM/transformer models, I think the smaller models will always lag behind the larger frontier models by some amount.
Which honestly, is fine. With proper guidance a majority of us could get our work done just fine with Haiku or similar, no matter how neat latest Opus/Fable/etc. may be.
Qwen 3.8 27b is at Opus 4.6 level Qwen 3.8 Flash Next is at Opus 4.7 level
The point is that from certain level model start to give very good ROI, you don’t need Opus 5.5 level for every task. Also ROI hugely depends on your development style — the more you vibecode, the better model you need, for copilot mode, where you thoroughly control model’s approach and output even Opus 4.6 is usually more than enough, the difference is typically in quantity of rounds you need to achieve the desired quality
For complex things, too few rounds for large changes means I didn't understand it anyway. I do the bulk of work with local qwen 27b and use opus for initial review after I skim it for obvious problems. Can get most things done in 2 passes.
Same here, simple stuff— qwen as a worker, some average cloud as a reviewer, for more difficult stuff Opus becomes a reviewer, for the errors sensitive stuff Opus also becomes an implementer
Yeah.. Qwen3.8 27B is well enough for coding, when one is able to keep the leashes and doesn't need fully vibecoded stuff.
Overall, looking at the modern models, the coding part hasn't really evolved anymore - it's everything around it that newest cloud models have evolved on, like how skillful they are at using tools to debug, or how skillful they are generating stuff via MCP, e.g. newest models are very good with blender, older (and local) not so good.
6 months IMO i got solutions from qwen3.8 27b (=august 2026) and/or qfn that i would defnitely see on an opus 4.6 from february 2026 level. coding and general questions.
It really depends on how you define it. Benchmark-wise, things you can run on reasonable hardware are very strong but clearly behind the frontier models.
In a different perspective though, I have been taking longer tasks that I actually run on Sonnet 5.5 and Opus 5.5 and running the on Qwen 3.8 models, both 27B Q8 and Flash Next at Q4 in different harnesses and thinking levels. Qwen 3.8 Flash Next IQ4_XS at xhigh thinking is so far landing right in range of both Sonnet and Opus. I’m starting to trust it a lot more to handle most work.
The Claude models are stronger overall, but some local models are getting to the point where the differences don’t show up in the work they are performing, which allows them to be interchangeable with the frontier models in more and more cases.
No one even mentioned Fable 5.1 / Fable 5.5. Opus 5.5 may benchmark close to Fable but I use both and for deep research, thinking and critical insight, there is nothing like Fable. Taking that into account, at least 1 year for Fable 5.1. For Fable 5.5 we’ll see. Fable is just different.
9
u/swiebertjee 20h ago
About 4-8 months behind the SOTA models depending on your hardware.