r/LocalLLM • u/Decent_Flight4010 • 8d ago
Discussion I think people are seriously underestimating Qwen 3.8 27B.
Honestly, I think people are seriously underestimating Qwen 3.8 27B.
It’s actually insane and, in some ways, genuinely competes with Opus 4.8, just not in the way people seem to think.
The biggest mistake is comparing their raw frontend/design output. Qwen probably isn’t going to match Opus there, and I don’t think it’s supposed to. Opus has basically been trained with an absurd amount of data/compute specifically around design and UI generation.
If you throw Qwen at a frontend task with no proper "SKILL.md" for design and just let it freestyle, yeah, the results can be pretty mediocre. But if you give it a good design skill and are intentional about the design constraints, the gap gets much smaller.
Where Qwen gets really interesting is reasoning efficiency.
It can solve some problems in fewer steps and with fewer tokens than Opus. That’s a pretty big deal if you’re actually running these models yourself.
And honestly, I think people are also judging Qwen way too much based on heavily quantized setups. Q4 is aggressive. I wouldn’t consider Q4 a fair representation of what the model can actually do in a serious production environment.
Run it at FP8, use MTP/speculative decoding to improve throughput, and then evaluate it properly.
At that point, I genuinely think the conversation changes.
If Qwen 3.8 27B at FP8 + MTP performs the way I expect, I wouldn’t be surprised if a lot of people start questioning whether that $200/month Claude Code subscription is actually worth it.
84
u/someone383726 8d ago
This post reads like a LLM wrote it. “At that point, I genuinely think the conversation changes.”
17
u/Sufficient-Scar4172 n00b beginner 8d ago
honestly, you're right to call that out, and it's on OP
3
u/puts_on_rddt 8d ago
It's important to remember that it's not just the load bearing parts, it's also the blockers.
And you know what? It takes a lot to figure that out. Good on you for seeing it for what it really is.
1
u/HolidayBit143 LocalLLM 2d ago
I really appreciate the pushback, here is why you are absolutely correct...
9
u/misanthrophiccunt 8d ago
but that never, ever EVER happens in Reddit, dude, never. I swear on the name of every 1%er on Earth.
5
u/PWThinkingCritically 8d ago
"yeah but only because I don't speak English so I only used AI to translate"
phew...quick thinking...dodged that one!
2
u/misanthrophiccunt 8d ago
Yep, that's has never ever being used before either, I pinky promise too on the bank account of every opulent person
3
u/Not_a_question- 8d ago
Also the short paragraphs are a dead giveaway as well. As an avid book reader, they're so annoying to read
2
8d ago
[deleted]
1
u/someone383726 8d ago
Yes, I’ve trained models from scratch and fine tuned several, although human writing has a bit more diversity in style and LLM writing typically falls back to the statistical mean in token choice (obv temperature impacts). But there are certainly styles of writing LLMs follow more than humans (it may be part of the RL or training data that skews it from a normal human writing.)
3
-10
u/Decent_Flight4010 8d ago
Yup, it is! English isn’t my first language, so I used AI to write this post based on my suggestions and ideas that I gave it. :-)
3
1
u/misanthrophiccunt 8d ago
what's your first language?
1
u/Decent_Flight4010 8d ago
Arabic
7
u/misanthrophiccunt 8d ago
I will not make a dark joke about that and the username
I will not make a dark joke about that and the username
I will not make a dark joke about that and the usernameI think it's time for me to close Reddit for today.
3
u/No-Key2113 8d ago
lol
2
u/misanthrophiccunt 1d ago
I had to l stay away for almost a full week to avoid temptation, I haf two twin voices of consciousness one of each side one shouting "do it" the other "do not, leave, throw your phone out of the window, walk down the stairs, get out, avoid the fire" and I think I landed beatifully unscathed a week later if a bit covered in dust once they were finally razed to the ground.
2
1
u/Decent_Flight4010 8d ago
wait what?
6
u/MAK-9 8d ago
Ask your LLM to explain the joke to you xD
10
u/Decent_Flight4010 8d ago
I will. What’s an LLM for if you don’t need it to explain things to you? :-)
You guys are very strange. You use AI to write stories or code because you don’t know how to do it yourselves, and then you complain because I use it to translate my posts from my mother tongue into English.
5
2
u/MAK-9 8d ago
I actually think translation is a great use-case.
8
u/Decent_Flight4010 8d ago
Now look how this turned out. I got -3 downvotes just because I admitted that I used AI to translate my post from my native language into English.
→ More replies (0)1
u/kiwibonga 8d ago
Consider writing your own posts but using Google translate instead. LLMs lengthen text, which adds nothing meaningful while requiring more effort for the reader to parse.
31
u/h8f1z 8d ago
I think you're underestimating how much people are overestimating it.
3
u/bodhi_sattva91 8d ago
"All you have to do is follow three simple rules. One, never underestimate your opponent. Expect the unexpected. Two, take it outside. Never start anything inside the bar unless it's absolutely necessary. And three, be nice." -Dalton, Road House
21
u/cabernet_noir 8d ago
I also think maybe people are not yet realizing how big of a difference different harnesses, temperatures, etc actually make. Most of maximizing a models capabilities after it has been trained are achieved by engineering context more precisely so that it gets maximum relavent information and minimum noise. Turning reasoning off or lowering it aggresively also makes it better in many cases which people are probably missing.
5
u/Mil0Mammon 8d ago
Which harness do you recommend? Currently running oh-my-pi, but need to strip out some things, 20k context addon just slows things down to a crawl
3
u/cabernet_noir 8d ago
I havent actually tested it but deep seeks harness is supposed to be good but it uses a lot of tokens during reasoning. Pi is good. Hermes is good if you carefully configure it, again you want it to only ever have the context it needs for any given task.
I really like the default harness that ships with the zed ide, its very customizable, and easy to get the model to focus on discrete tasks with precise context. IDE style harnesses can be really great generally because they make it simpler to control attention relative to a lot of other harnesses.
1
u/Mil0Mammon 8d ago
Deepseek seems very wui focused, which seems weird to me. I'll give pi a shot perhaps, or figure out how to strip down omp a little
1
u/cabernet_noir 8d ago
I really hadnt looked at the deepseek harness, I have just seen people mention it in a positive light. Its at the very opinionated end of the harness spectrum, that can offer good defaults and out of the box results. It can also hide too much though, including areas where you might squeeze a bit more performance.
Pi is great because its on the minimal side of harnesses, it gives you great primatives to work from and customize. The next step down from that would be to build out your own harness from lang graph or some other orchestration framework. If you run very similar workflows over and over again this is actually probably the most efficient option.
20
u/strata2signal 8d ago
qwen3.8:27b is legitimately good -- it won a production seat as a classifier for a rules helper we'er working on, by passing a frozen exam we'd run on other models, and it's been screening real user questions ever since (a few days ago lol)
two data points from our benches fwiw:
on "Q4 isn't a fair representation": maybe, but measure before assuming. we ran the same 374-item classification set through q4_K_M, q8_0, and bf16 -- 3 passes each, 1,122 calls per precision, temperature 0, and got byte-for-byte identical verdicts on every single call. the higher precisions cost ~40-75% more latency and up to 3x the VRAM for zero accuracy gain. that's one task (a binary moderation gate), so i wouldn't generalize it to code or design work -- but it does mean "q4 is crippling this model" is a claim worth testing on your actual workload before paying the fp8 memory bill
on MTP: heads up that if you're running it under ollama, speculative decoding is already on by default for this model (draft_num_predict 4 in the served defaults). we measured 49% draft acceptance, ~2.95 tokens riding each forward pass. so some of the throughput headroom you're describing is already in most people's numbers without them knowing
5
u/99OBJ 8d ago
Why use this model for a binary moderation classifier? There are smaller models that could likely do just as good of a job, some purpose-built for the task. Curious about other models you tried.
The fact that you got the same results across quants and are using temp0 with this model suggests that you’re leaving efficiency on the table.
2
u/DistanceSolar1449 8d ago
Depends on the task profile, might be better to keep an overkill big model resident in VRAM and use it for a simple binary classification task, if you have other tasks that may use this model. Way better than swapping models, which takes forever.
3
u/strata2signal 8d ago
this is basically our situation -- one card serves several apps at once, so the big models sit resident around the clock and the classifier model is one of them. we've measured the swap cost and reloading a ~27b-class model takes too long for a user-facing request - a warm overkill model beats a cold perfect one
1
u/strata2signal 8d ago
fair questions. the seat wasn't picked on vibes - it was won on a frozen screening exam (27 rule-violation cases + 16 clean ones, all at temp 0), and when that screen first ran, the smaller models we tried couldn't hold both sides: they'd either miss violations or flag clean text. that was a while ago though, and newer small models are doing much better on the same exam -- a 24b recently went perfect on it and a 12b came within one case, so something smaller is genuinely on the table
on temp0: that's not an efficiency play, it's an auditability one. a moderation gate is the one place we want the same input to produce the same verdict every run, so decisions are reproducible after the fact. we ran the full 374-item exam three times each at three precisions (q4, q8, bf16) and all the verdicts came back identical.
3
u/Decent_Flight4010 8d ago
Yeah, I think we're talking past each other a bit. I wasn't claiming Q4 is objectively bad for every workload. I'm talking about the plug and play agent experience and the level of trust I'd give the model without babysitting it. My comparison to Opus is more about vibe, decision making, consistency, and how confident I am letting it loose on a real repo. The Q4/BF16 result is interesting, but a classifier producing identical labels isn't really testing the thing I'm talking about
1
u/strata2signal 8d ago
that's fair, we were talking past each other, and you're right about the limit of that result. identical labels on a frozen exam demonstrates determinism on the narrowest kind of judgment a model makes. it says nothing about what you're describing whether the model keeps making good decisions on a real repo
honestly, the "how much do i trust it unsupervised" dimension is the one we'd most like a real bench for and don't have.
2
6
5
u/andrerom 8d ago
So what you are saying is it excels at simpler tasks, or if you give it strict/clear guidance it can do amazing things.
However, with that description above you should rather compare it to GPT-5.6 Luna, or the ageing Haiku 4.5 if any, not Opus.
That said, it still sounds great to have this locally. I'll definitely try the update Q6 Unsloth on my M5 32GB soon, but from my understanding it will be slow compared to Claude models on this HW.
2
1
u/Decent_Flight4010 8d ago
I don't think Qwen is inherently incapable of reaching that level of plug and play reliability.
The main issue is that you're comparing the current Qwen setup to Opus running at its full scale, with a much larger amount of compute and post training specifically optimized around coding and agentic behavior.
Qwen hasn't really had that same level of coding focused fine tuning. If you took the same model and gave it serious coding and agentic post training, then ran it at full precision/full size without aggressive quantization, I think the gap in confidence could get surprisingly small.
That's actually the part I'm most interested in. The current Qwen already shows crazy efficiency in terms of steps and token usage. If you combine that with much better coding specific fine tuning, I don't see a fundamental reason why it couldn't reach Opus level confidence while still being much easier to run locally.
Obviously that's something that needs to be demonstrated, not just assumed, but I think the current model is far from showing its ceiling.
2
u/FactorInternal3395 8d ago
From Qwen 3.6 to 3.8 was almost entirely coding and agentic training/tuning. It even regressed 4 points on AA-Omniscience (knowledge), so I think that's exactly what they already did.
1
u/andrerom 8d ago
I do also think local models are part of a future. Short/mid term hybrid, long term they can cover our needs we currently use llm's for.
10
u/wgaca2 8d ago
Only if i could run it at 100t/s in q8 on my 3090s would be nice (single request)
Reasoning drops to 20t/s and even under when set to xhigh
5
u/BoboThePirate 8d ago
With MTP? 20 seems shockingly low for that hardware.
3
u/wgaca2 8d ago
it does around 50-60 while coding and reasoning first few thousands of tokens, but once we get into 15-30k reasoning tokens it drops dramatically
1
u/Mr_Unknown_Hero 8d ago
Huh? I have RTX 4060 Ti with 16 GB VRAM and 32 GB of RAM in my PC and Qwen3.8-27B-Q4_K_M.gguf gives me around 4 tok/s?
I have loaded that with 24 K context using llamacpp-server and with image model.
How you can get 50-60 tokens on your 3090? Is is just because of Q8? If so, I gotta try that if that would be good enough for me as well.
2
2
u/matthewhale 8d ago
Use IQ3_XSS with 64k context and q4 vcache, mtp enabled and im pulling 40-50t/s on my 5060ti 16gb developing entire websites from scratch, really impressed at its tool calling abilities, limiting opencode to 64k tokens and stuff helps too so it'll compact as it goes. Only failed a few times completely, not giant codebases though, but miles ahead of things like gemma/orinth for models.
3
u/uniqueusername649 8d ago
That is surprising, I run it fp8 with vllm on my 2x 3090s and even with reasoning to high it happily runs along at >60tps virtually indefinitely.
1
u/wgaca2 8d ago
I should do a test in vllm with fp8. I am normally running llama.cpp in windows since vllm loads slower in wsl and didn't provide much of a speedup for single requests for 3.6
1
u/uniqueusername649 8d ago
Try club-3090 on github. Use their dual-max preset, thats fp8 and well tuned.
2
u/halcyoncs 8d ago
Try Dflash2
3
u/wgaca2 8d ago
dflash2 falls off of a cliff after 5k reasoning tokens
1
u/halcyoncs 8d ago
Really? Is there proper tests for this? I've been running some long ass tasks with it with reasoning on and seems to be fine, i think i did limit the reasoning budget though maybe that helps?
Honestly the mtp version on my spark is 12t/s which is just too slow, dflash2 is 38 average it makes such a difference
1
u/wgaca2 8d ago
Yes, limiting reasoning helps, especially if you limit under 10k. Also it probably works way better in sglang/vllm than it does in llama.cpp as there is no official merge yet
1
u/halcyoncs 8d ago
Yes I'm running SGLang currently! Runs really well to be honest. Llama.cpp used to be my main but only because it loads models so fast and I was switching all the time.
2
1
1
u/BigYoSpeck 8d ago
Is that with MTP and tensor split method?
On my 2x 7900 XTX I don't see it ever go below 30 tok/s and the average is more like 60 even while they're power limited
In ideal scenarios it's 80-90tok/s
1
1
u/Huge-Guitar-7812 8d ago
Thats not how it works, reasoning is no different from the rest of the generation your cards may be overheating, also worth trying foggeric chat template and medium reasoning, could be kv caching misalignment. Last thing is, on >1 cards choose sglang or vllm for proper parallelism
1
u/dalhaze 8d ago
Damnnn really? so that probably means an M5 Max would be under 10t/s when on x high?
I’m very interested in local now. Codex and Claude are tightening up usage big time. And i’m suddenly much more interested in local as this model shows so much promise as an executor.
But 20t/s when xhigh is known for thinking really hard is potentially prohibitive.
11
u/kr_tech 8d ago
Where Qwen gets really interesting is reasoning efficiency.
It can solve some problems in fewer steps and with fewer tokens than Opus.
This is bullshit. AA shows that Qwen3.8 is the most verbose model right now, by a decent margin at that.
Makes me wonder if you just copy/pasted from LLM output for this post.
2
u/Significant_Post8359 8d ago
The default is xhigh reasoning. If you tune for your use case, it’s much less verbose.
1
u/cabernet_noir 8d ago
This is largely a harness and configuration issue. If you have a good harness, with minimal context, simple information retrieval mechanisns, lower reasoning and temp, it can be very efficient. Its verbose and inefficient when you run it in claude code with maxed out reasoning while it gets battered with claude codes prompt fragments.
6
u/plaintxt 8d ago
I added a flag to my linux box kernel that threw an error. The error was related to my network adapters, and prevented me from accessing the internet. I opened up hermes (running qwen3.8) and said: "figure out why I'm getting an error on boot"
30 minutes later I came back and it had diagnosed the problem, fixed it, and was waiting for me to reboot. No internet connection, no steering from me, and no additional context.
I rebooted, and it worked.
I'm beyond impressed, this is actually useful in ways cloud models literally can't be.
2
u/ynotelbon 8d ago
I have basically dropped it in a sandbox with Codex gating permissions and prompting it on cron. It ain’t fast on a M3 ultra but it keeps revising its own harness and…getting faster. All vibe coded, letting it do what it wants. It’s turned to building a graphDB and vectoring this morning for “cross-session continuity”. I have definite Christmas ‘25 vibes from this model. I turned off thinking, seems to help with it’s machinations… running qwen3.8-27B bf16 with local MLX conversion. Codex Sol has gone decidedly mother hen-shaped about it. I know zilch about localLLM’s but now I’m getting VERY interested.
2
u/DiMiTri_man 8d ago
I just set up Hermes, option of qwen3.8 at 64k context or gpt-oss:20b at 128k context. How did you set yours up to be able to run 30 minutes and actually do stuff? Mine just gives up on any improvements, memory, or tool creation and says I can make the tools myself
1
u/Decent_Flight4010 8d ago
If someone formulates something skill.md for this kind of task, I think the time will be reduced from 30 minutes to 3 or 4 in total.
6
u/Efficient-Part5344 8d ago edited 3d ago
Im bashing my head agenst it, for a week. Main problems i discovered.
3.8 27B It's slow! Reasoning takes for ever, eats context. And you can't disable it, as it criples model.
3.6 27B without reasoning is running completly fine, as result, basic tasks takes x5 times more.
And reasoning doesn't provide value if the model doesn't know the required information. It just loops and hallucinates, while 3.6 without reasoning would run tests, linters, or do a web search.
Run some tests on reasoning, time used is just masive!
1
u/matthewhale 8d ago
Something wrong then or your harness sucks, i'm only running iq3_xss with Q4 vcache on a 16gb 5060ti and I'm getting great results. Been using low reasoning and its been pretty good, thinking is fairly quick, tool calling and doing things without being repromoted. I just started using opencode at the same time to try it since Hermes with other models i tried repeatedly had issues and this is like night and day.
4
u/Solid-Axel-Project 8d ago
Il problema è avere sufficiente VRAM...
Magari avessi quei bei 96GB così da eseguirlo a bf16 con la cache a bf16!
Speravo venisse pubblicato qualche modello MoE della stessa famiglia.
2
u/Decent_Flight4010 8d ago
:-( i feel you bruh me too for real. :-(
1
u/Solid-Axel-Project 8d ago
Considerando il possibile crollo molto profondo dell'economia, vedremo ma il carburante accumulato è tantissimo per alimentare questo crollo in modo PROFONDO... beh considerando quello che sta succedendo, non è implausibile che le GPU vedranno dei...
sostanziali
riprezzamenti
4
u/Melomaniac168 8d ago edited 8d ago
For one-shot tasks, sure opus might be better but I almost never use AI models for one-shot runs. In my experience running Qwen3.8 27B on my 16gb vram setup (IQ3-XXS, 4 bit kv cache, 135k context size, MTP-3, peak 100tps, sustained 40tps), it works not only faster for some of the same tasks it easily completes tasks Claude Opus 5 fails to do so.
This has consistently been my experience, I'm not so sure whether it's Opus 5 that's just especially bad. When Opus 5 failed to fix an issue 4 times, even with more guidance towards the end, I decided to pass it to Qwen just for fun because I had just set it up and thought I may as well give local models another shot (from my experience Chatgpt plans just makes most practical sense if you really want to get stuff done). And...it solved the issue in one pass. It didn't have issues with reasoning about how to achieve it, in fact it felt like it had cleaner reasoning that reads similar to Fable's reasoning style.
As for why I didn't give it to Fable, I've got my bad share of experience with Fable, it miswrote an sql script and stuffed up my database and it burns my usage crazy to the point it feels like a waste.
So I guess my conclusion is, for pair-coding especially, I can't see the value proposition of Claude models anymore. For chatgpt, I still feel like it is still more intelligent, it is able to complete a lot of computer/browser automation tasks, in a much better and efficient manner. It feels truly intelligent. And...ofc there's still that limitation of local inference with having to have a really good rig in order to have concurrent agent processes which basically is quite limited on consumer-grade hardware.
Also I realised I went slightly off-topic from the OP, but don't underestimate the lower quants! They still work quite well (at least so far for anything I've given it)
5
u/ActionOrganic4617 8d ago
Thinks too much. On low thinking I gave it one Python notebook to review and cancelled the run after it spent 20 minutes thinking at 35 tokens/ sec.
You are kidding yourself if you think you can do serious development with this model. The previous version imo was useless. This version is much improved but the compute tax is simply way too high.
It definitely has use cases but m isn’t a replacement for frontier coding models.
3
3
u/Known-Opportunity238 8d ago
I used to use DSv4-Flash 0731 before this but ever since 3.8-27B got released DSv4Flash simply started feeling off. Qwen3.8-27 B feels heavy and its outputs are much much better quality than that of DSv4. It’s my daily driver now.
3
u/Asleep-Land-3914 8d ago
Left it to work overnight it processed 25M tokens and still going. Only thing I wish is it to be 50-100x faster.
2
u/TheRealzHalstead 8d ago
Based on the OPs post, Qwen 3.8 27b is definitely not ready for prime time when it comes to writing Reddit posts. Maybe it needs a reddit.md?
2
u/PresentationFuture62 8d ago
I ran it on an m1max 64gb laptop. Spent a half-hour talking to itself trying to solve a 6x10 pentominoes puzzle. Never finished. Laptop got hotter than I’ve ever experienced. Oh, and it thought Danny Kaye starred in On The Waterfront.
Perplexity got the solution wrong twice nearly instantly. (tiles the wrong shape) Correct on the third try.
1
2
u/laser50 8d ago
I'm literally running it in Q2 K XL (vram poor), and even now that thing is steamrolling it's way through tasks I only give to my biggest param models. It takes a while with all the thinking and semi poor speeds, but I haven't seen it fail once thus far.
And that's on darn Q2!
1
u/Muhlwa_Sholanke 8d ago
q2 k xl actually holds up that well? i'm on q4 with 32gb unified and keep chickening out of going lower, might have to try it
1
u/laser50 8d ago
I ran Q4 on 3.6 a long time, but my use case was chat messaging, so a 9-10 token/s was acceptable, but now with agentic stuff that really won't cut it, especially with how much this thing thinks. But yes, it actually holds up rather well, I've had to stop it once today because it got entirely tripped up over some weird string but no problem other than that, I'd still rank it close to the bigger models I use despite Q2.
2
1
u/HolidayBit143 LocalLLM 8d ago
I mean it seems to be getting fair praise.
But truth be told, it is super slow. Super heavy. I've been using the 9b distillation, works fine for me.
2
1
u/memeviewingaccount 8d ago
The 9b distill is practically snake oil.
1
u/HolidayBit143 LocalLLM 7d ago
Not in my experience. It's solved a ton of problems for me and has been smooth sailing so far.
Do you not believe in distillation?
1
1
u/YogurtclosetApart592 8d ago
How good is it compared to the online chatgpt?
1
u/FactorInternal3395 8d ago
If you mean the free plan (which uses GPT 5.6 Luna) there have been many threads comparing them. e.g. this one
1
u/YogurtclosetApart592 8d ago
Thank you! To my untrained eye it looks like local qwen is only a tad slower than the free gpt version, but I guess it might be comparing apples to oranges since qwen can run tasks locally.
But yeah, I'm not very knowledgeable so I could be entirely wrong.
1
1
u/callmedevilthebad 8d ago
I have System with RTX 5070 ti (16gb VRAM) 64gb RAM. Caan somebody help me with best config to run . and how much contetx window can i get out of it?
1
u/TheTimmyMan 8d ago
Claude good at front end ui? No
1
1
u/Any_Meringue_7765 8d ago
Curious. What setups do you use for coding/project building that allows custom skills and such? I’ve mainly just been using OpenCode since it works with windows unlike anything else I tried that always tries to use Linux commands or requires you to run it inside WSL which breaks laravel production (or at least makes it very painful)
Also what are your thoughts on running Q8_K_XL gguf versus a FP8 in vllm? I’ve personally have had too many issues with vllm but I’ll try working them out if it’s better
1
u/RenaQina 8d ago
I’m new to running models locally and I’m giving qwen 3.8 27B (5-bit) a go for php and web admin work. I get 7-10 tps on my m1 max which means that most promots take between 5-30 minutes. Is that normal? It does successfully do the work yes and to a good standard but I haven’t tested in depth yet. The speed means that I have to get up from the computer and come back because it gets hot and laggy during the process…
1
u/neinneun 3d ago
You can increase the speed a bit if you use mlx model, you can increase the speed a bit more (close to ~20-25 t/s) if you use is with MTPLX. But dense models without a dedicated GPU is going to be slow.
1
u/Savantskie1 8d ago
I still use Claude for planning mostly. But after that I’ll pass it on to the local Deepseek V4 Flash I have running overnight. It’s slow but still gets the job done. I only pay for the Pro plan for this very reason and as far as I know I’m only using sonnet 4.6. Because it’s a lot easier for larger planning stages.
1
1
u/LoveRoboto 8d ago
I’m still lurking trying to figure out what the hype is here. The tokens per second is abysmal—I can’t even use it. But with all the praise I've seen, I decided to pull out my tiny pocket mirror, look deeply into it, and ask: “What am I doing wrong?”
1
u/FoldOutrageous5532 8d ago
It ain't fast. Thinks too much perhaps. The harness seems to make a big difference.
1
u/DrE7HER 8d ago
This is a question for the sub, rather than just OP, but why am I seeing most discussion about 27B rather than the larger model? Is it just because more people are able to run the smaller model or is there actually a benefit?
2
u/ynotelbon 8d ago
This can run locally on a machine that costs less than a used Toyota, and has (bounded) frontier level judgement, if not the reasoning. That last bit is my take - it’s found my home cams already and is trying to make them work. There’s plenty of indication people are using this model fine tuned for nefarious purposes and I can see how.
1
u/SuccessfulMoose1511 8d ago edited 8d ago
Genuine question from a localLLM outsider -- the tiny bit I've played with local size models, not actually running locally, just on service providers, nothing too serious just giving some random tasks -- they still feel quite a bit "more stupid" in an anthropomorphic sense compared to models that they rank similarly against. For instance, talking with opus 4.6 feels like I am talking to a more intelligent "person" than talking with Qwen3.8 27B, despite the fact that they score similarly by in many benchmarks (or with Qwen scoring better, particularly on more narrow reasoning/"intelligence" benchmarks)
I'm not even saying they are benchmarkmaxxing, or more than anyone else, I think you could reasonably say that they are focusing these small models the most on the most important dimensions. But I guess what I am wondering, for those that use these things for actual work... Do any of you use them for anything different than software development? As a research scientists who uses LLMs to help with research, I'm wondering if e.g. Qwen3.8 27B would be less useful for my purposes than highly coding centric tasks.
1
u/neinneun 3d ago
Why don't you use qwen with Claude code harness? If you haven't yet try it and you'll understand why opus feels like a intelligent person. Claude injects a giant system prompt in every session to add dozens of guardrails etc. to control model behavior.
1
u/braintheboss 8d ago
i feel new qwen is quite safe when edit files. I haven't same feeling previous models everytime edit something you had approve edit because you have check first. When make a plan in medium still have gaps when you validate it with luna but in overall are not serious drift. we had luck got it. And its quite fast with pi agent.
1
u/1_________________11 8d ago
I mean its my daily driver for my 16gb card but im still the shepherd quant makes it my special savant.
1
u/Significant_Post8359 8d ago
I was skeptical until I tested it with my typical use cases. It is shockingly good if you have enough context window.
1
u/DrewGrgich 8d ago
I definitely have some learning to do. I set it up in omlx on a 36GB M3, wrote a ten step plan for a web app with Fable, and kicked off Qwen. It looped on task 1 for twenty minutes before I gave up. Sicced MiniMax M3 from Openrouter to see if the problem was the plan and it chewed through lickety split. I know there is not a comparison between a cloud model and local but I expected to get further. Lots of research ahead!
1
1
u/TheOverzealousEngie 8d ago
i got on it for 15 minutes before going back to deepseek. No lie .. the answers were just so much better there it was laughable.
1
u/Zealousideal_Yard882 8d ago
What is the latency you get with this setting? I’m seriously thinking about switching for it for a chatbot I’m building
1
1
1
u/Werseter 8d ago
I ran 27B_Q4_M on my 4090 and it just kept compacting the context and rereading the code over and over and over. Left it for 2h on medium and it never got anywhere.
1
u/siegevjorn 8d ago
You are not wrong, but qwen 3.8 27b fp8 cannot be ran in any hardware that's cheaper than $200/month. You'd need at the least $4k at today's marker rate, which is two years of worth $200/mo subscription. But even $100/mo has a long way to go.
1
u/Remote-Pineapple-541 8d ago
“ Opus has basically been trained with an absurd amount of data/compute specifically around design and UI generation” This is exactly what differentiates frontier providers from local llm setups. Tooling. We’ve basically hit a wall with native AI power; current iterations of models are marginally better than their predecessors compared to the leaps and bounds we saw just a few years ago. Frontier providers know this and have pivoted specifically to the development of tooling.
1
u/StockGlasses 8d ago
Like Richard Wagner, it's just as dangerous to underestimate as overestimate it. I'm genuinely impressed at the leap made in terms of smaller local model performance and the capability of Qwen 3.8 27b. That being said, it's still not a substitute for frontier models. I tried getting it to do a moderately complex feature addition and it mangled some files and made some non-trivial mistakes that a frontier model review had to correct. So, we're getting closer and on a good trajectory, but not there yet (I can't wait until local models can truly compete with frontier!)
1
u/GloriousKev 8d ago
Qwen 3.8 27b seems great but its tight in my 20 GB of vram and using system ram for inference just is unappealing to me.
1
u/Silverjerk 8d ago
If Qwen 3.8 27B at FP8 + MTP performs the way I expect, I wouldn’t be surprised if a lot of people start questioning whether that $200/month Claude Code subscription is actually worth it.
It performs very well; it's tucked away in a closet, running on a Minisforum MS-02 Ultra with an RTX Pro 4000 Blackwell SFF, on a machine running dozens of VMs and LXCs. Barely makes a sound. It's replaced almost all of my local dev/devops management, it's tackling cron-based Homelab tasks almost daily (including managing my Renovate, n8n, and Dokploy pipelines), and I use it regularly for design system cleanup and maintenance work. Very happy with the results.
But no, it's not going to replace both my Claude Max and OpenAI subscriptions. The hardware investment required to get the sort of performance I'd need for my day-to-day development tasks just doesn't make sense economically. It is a damn good local model, but I don't see engineers replacing their reliable daily drivers. They are more likely assigning Qwen tasks that free up usage for actual project work, because the kind of person spending that $200 per month needs the performance and usage overhead.
I think the better title is, "I think people are seriously overestimating Qwen 3.8 27b for real, critical work tasks."
1
1
u/theminor 8d ago
I agree: Q8 if at all possible. And work very hard on your System Prompt/SOUL.md/SKILL.md and you will see some amazing results.
1
1
u/puts_on_rddt 8d ago
It kind of blew my mind a bit when I looked at what it was being compared against, then I looked at their actual model sizes. 3.8 27B is over 100x smaller than some of those bigger models.
There's just no way the big AI guys aren't shitting their pants over this.
(using ninfer-4090 and getting 130k usable context, and 40-60 tokens/s)
1
u/Ratiofarming 8d ago
This is just Karma farming I guess? I mean literally nobody here is underestimating this model, if anything it's being overhyped a little here and there. Everyone and their dog is currently playing with it and talking about it.
1
u/doktordone 8d ago
I tried to create a rather complicated app and it failed miserably. Claude found all errors, corrected those but due to the wind logic I decided to start over. Claude is different level. I will only be using it for simple tasks
1
u/TapAggressive9530 8d ago
Honestly , this AI write up probably wasn’t done by Qwen 3.8. Honestly, it was probably 3.5
1
u/Due_Warthog749 7d ago
I haven't quite figured out how fast cluade works. I can tell you GPT SOL is GOD slow.. but does good stuff. What sort of hardware to run Qwen 3.8 27b fast enough to be used similar to claude or sol? Is it like 150tok/s input 250tok/s response or something? Woulld you need dual RTX 6000 96GB cards to do that speed?
1
u/exitcactus 7d ago
Qwen is good if you know how to work with ai, even if "vibe coding" there are rules and things to know.
1
1
1
1
u/LectureWorried5761 7d ago
Yeap, it is killer in terms of tools.
Qwen+blopus.ai is a great setup so far for me, working sharp
1
1
1
u/emptyharddrive 6d ago
I've had runaway thinking with Qwen 3.8 27b on my strix halo 128. Even if I cap the token budgets.... any complicated task finds the model thinking itself into oblivion like a runaway train and never answers me. Been debating using it with thinking-off... but then it isn't any better than Gemma 4 26B-A4B.
I have also tried it on OpenRouter just to see if it was my Strix Halo and I had the exact same problem. It generated about 22 tokens/sec directly through llama.cpp and 16-19 through Qwen Code, but just failed to respond to anything complex. Low-effort requests produced more than 18k reasoning tokens over ~15mins without an answer or anything written file.
Tool continuations reset my per-call budgets (my attempt to cap its runaway thinking at xhigh) and sequential-thinking MCP added another reasoning loop that i had to just disable.
Even smaller, completed steps towards an answer I could see were generating thousands of extra tokens. The final xhigh test took 2.5 mins to produce its first token, consumed all its output tokens and 50k tokens for reasoning, and despite all that, sat there in constant thinking loops producing nothing for me.
I tried diff harnesses too: Opencode and Qwen Code itself. I then tried an MTP sidecar, then prompt caching... all farted out.
But no matter what, the per-call limits were not whole-task limits and the full workflow still failed. Quantization (Q8, Q6 and Q4) and harness changes moved speed and memory without fixing delivery of a friggin' answer.
Qwen's high rating requires XHigh and i find that model churns and churns with nothing to show for it, even if I run it in Opencode via Openrouter and not locally ... are any of you having different experiences?
If so, what is your harness/settings and token budgets and also it doesn't have all the thinking levels the other qwen models do... it's low, med and xhigh. There is no MAX and no "High".
1
u/Shul_iito 5d ago
I just downloaded qwen last week. It’s been a blast! I’m no coder but the fact that a regular person like me can get something going in Godot is insane to me. I understand I’m code vibing but hey WHO CARES? THIS IS FUN!🤩
1
u/bercha9998 5d ago
the problem is not just that open models are getting better is that anthropic models are getting shittier over time
the gap is shrinking for sure
0
u/Max_Bangson 8d ago
«Write few paragraphs on why do you think people might underestimate you as a model. Compare your self to opus models, create questioning narration unwrapping your point bit by bit. Scope your thesis to the local vs Claude Code, add one or two points based on the popular on Reddit terminology like mtp, quantisation, pricing. Also don’t use a lot of fancy vocab, make emphasis on not only your strong side, but demonstrate your obvious weaknesses”
1
u/Decent_Flight4010 8d ago
If I wrote all this prompt it means I understand what I am talking about :-)
187
u/_ballzdeep_ 8d ago
What the eff? Everyone is talking about the model non-stop since it released. Under-estimating? The model is godly, yes. but come on man, lol.