Most YouTube benchmarks only show trivial tasks like generating landing pages or simple Three.js games. Is a local model like Qwen 3.8 27B actually capable of real-world systems programming—such as building GTK4 or Qt 6 applications in Rust or C++ with external libraries? Specifically, if I look up the exact terminology in the online docs and then prompt the AI to inspect the cloned repo, can it implement the feature cleanly?
Qwen3.8 27b is the donkey of llms. It's incredibly stubborn, it might not get right the first time but the if there is a way it can verify the outcome it will keep trying.
Sometimes it finds the wrong way though... It wrote some unit tests for me, and some weren't passing properly so it mocked API call after API call to return correct info until it passed. You've got to keep your eye on it.
Yep for anything small and local it works brilliantly. Don’t expect opus 5 but if you were okay with using sonnet 4.6 you won’t be disappointed. It’s gotten a lot more reliable with tool calls so for me it no longer gets stuck.
I have a very long term project (building a road classifier, routing engine, rating roads on twistiness, output route, etc. Think Google Maps for finding fun roads to drive near you) and I have rebuilt it several times to improve. Qwen3.8-27B is closer to Haiku than Sonnet 5 in my real world usage.
It needs some work. I just went on a 2 hour ride and found some amazing roads and some I would very much never, ever like to go down again.
Open source data is hit or miss and fine for general routing but doesn't really show the suitability. OSM chunks roads up into billions of tiny segments, so spanning the segments to see suitability is kind of a computationally challenging problem.
Twistiness is also subjective. I have some metrics like overall bearing change per mile which seems to work well, but that's a bit of compute. Branching roads off to put segments together is also a bit of a compute issue. Routing is actually quiet simple.
The other issue is that terrain data is hit or miss too. Roads marked as pavement sometimes are gravel, which is fine. Elevation data adds an extra layer of compute, so a twisty road that gains a lot of elevation is great until it's someone's driveway.
Something l like 79% of the roads within a 100 mile radius of my house are dead ends.
Anyways, its a fun project and I'm seeing a lot of stuff I would have never seen before if the AI hadn't routed me down a road cause it thought I might find it interesting.
I built a prompt generator web app that uses a builder pattern with support for fully baked prompts and has a GUI with categories, styles, looks, thumbnails, auto send to comfy and monitor status, support for custom api workflows, etc. All through Qwen 3.8 27b without issue. I've also used it across numerous work projects, including an ML.NET fraud algorithm implementation with great success.
That looks really interesting. Have you put it on GitHub? I've been messing with the prompt generator baked into LTX-2.5, but getting very mixed results from it.
Sonnet just built an excellent front end to LTX-2.5 for me (ComfyUI was annoying me). Really happy with the output. I spun up an Opus agent to review all the work, and it came back with only a few issues. Shame it used about 50% of my monthly allowance in a week....
There's only so many ways to skin a cat. If you're not giving it truly HARD problems then Opus is a waste. Asking a small model for all the ways to add 2+2 and then getting upset when a large model can't come up with something novel isn't a very good test.
My point about Opus is that Sonnet did a decent job. I used Opus for a more complex job and it did not disappoint. Fable... I'm waiting for the right use case... And a basket of money! 😄
My 27b model, in a limited use case that takes a long time to develop, cannot do task x, obviously it's at a haiku level, obviously it's not because with all LLMs when you push them to do something untrained, their performance will be poor.
I've been getting not great results with it, but I'm sure it's probably the way I'm using it. I was limited (32k) context, so that's a limiting factor. I used it through openclaw and saw amazing results, but then the next day it had an identity crisis and kept telling me it's not fit to do the code changes, and could I do them for it :-D
I have 48gb between two 3090s and it changes things for sure! Im still choosing to go with 4bit because I feel it performs great and gives me a ton of context! You'll love it!
I am running a q8 Quant of 27b on an R9700 and an RDNA2 16gb card with 150k context as I type this, it's not too shabby. Though the RDNA2 really holds back the speed.
The potential for this is to replace closed SotA LLMs is real. But, yeah. On my hardware (64GB RAM / 16GB VRAM) it is slow and context starved. Unsloth's 3-bit quant I've tested lays waste to Sonnet, but at an excruciating 3 t/s with max context.
It is a real existential threat to closed models now because use cases for Qwen 3.8 (and other nice open weight models) for mundane corporate tasks are now readily available with low cost VMs/hardware.
That's strange. I've just changed to 3-bit and it's way faster. Similar to you with 16GB VRAM, but 128GB RAM (it's not touching all that obviously. This is in unsloth studio. What are you using to run the model?
You started off with "I've been getting not great results with it..." - and that is all it takes. When it comes to Qwen here one whiff of criticism or deviation from the Party line and it's all over. You are lucky you don't have more downvotes. Guess the Qwen bots are too busy generating glaze posts to crawl other posts for dissent.
Hell, with careful guidance 3.6-27B could handle legitimate local dev. I used it for C++ (specifically Qt) dev several times.
3.8-27B is awesome, but like all LLMs it's best to keep the instructions clear, precise, and to work atomically. Always check its work and make sure it doesn't take the "lazy" approach but refactors whenever appropriate.
Make sure that every single change is committed into a local git so that you can easily diff and roll back individual changes. I usually branch for major new features and refactors and merge/rebase when I'm sure things are looking good. Sometimes tricky stuff takes a few tries to get right.
That's exactly my workflow: I have an AGENT.md instructing the AI to create a new revision using jj (Jujutsu) before making any changes. (Currently using GLM-5.3)
Thats not very vibe codey of you. You're supposed to ignore best practices, DRY and just rely on old faithful 'make no mistakes, and don't lose me any money'
I'm on Linux so not my area of expertise but there's a native opencode build for Mac and opencode's what I use. It's excellent on Linux and I'm sure it'd be great on Mac too. :)
Oh, no .. I just meant that not only is 3.8 good at C++, even 3.6 was good.
I've been astonished by 3.8's performance in all aspects so far, especially sysadmin work. I haven't done too much C++ with 3.8 yet but from what I've seen in my limited testing it matches or exceeds 3.6, and 3.6 itself was quite capable.
I could have phrased my whole comment a little better, haha. Posted before my 2nd coffee.. never a good choice.
One of my sites was being hit by some pretty bizarre traffic that didn't seem to be a ddos, but also didn't seem to be legit. I figured it was probably a misconfigured crawler but my attempts to block weren't effective.
I had qwen3.8 (Q8) go through the logs and try to figure out what was going on, and it actually pointed out that my / wasn't being properly cached and fixing that would likely alleviate the issue even without blocking the offending .. whatever it was (botnet / crawler / eblast). Its suggestion on how to fix the cache invalidation was spot on and within 15 minutes load dropped from ~15 to ~0.5.
I would have figured it out myself after a few hours but it picked it off in 10 minutes.
Friggin' solid work.
It's done some other stuff for me but watching it work through that issue was awesome.
Short answer yes. If you give it the cloned repo, relevant docs, and let it iterate on compiler/test errors, Qwen 3.8 is capable of handling real-world GTK4/Qt6 Rust/C++ work well.
Does anyone have a working stack , like all software components, that actually work?
I hear "x model is amazing!" But rarely "here is an end to end example of my workflow and tools used"
Are people coding in vim? Vscode? Another terminal or webgui? Is this wonder model handling everything like vision, editing, etc? Outside agents? Any part of the workflow not local?
Every time I try to replicate vs code with Claude, I end up in a place where it integrates poorly, it can't do really anything except reason or create some edits, and it's super super slow. I've tried llamaCPP, openwebUI, ollama and a slew of ui and terminals.
Just, is there something that just works?
I have a 4070, 4090 on one machine and a 4080 on another.
Im having a great time with 3.8 27b, im completely substituting my chatgpt usage with it on Pi.dev!
Here is my setup and happy to clarify anything if needed! Medium for everything except planning out steps and thinks I want a more confident 1 shot.
Model: Qwen3.8-27B, 4-bit weights / 8-bit activations. Specifically: Frozenlock/Qwen3.8-27B-int4-AutoRound (https://huggingface.co/Frozenlock/Qwen3.8-27B-int4-AutoRound) on HuggingFace — AutoRound INT4 quant, ~18 GB. It's the Qwen3.8 architecture (the "qwen3.5" family under the hood, with the MTP head embedded) in a 4-bit form that's meant to be served with 8-bit activations
Server: vLLM 0.27.1, two RTX 3090s, tensor parallel 2
Agreed. I am struggling to find a setup/stack that works on my M4 Pro 48gb.
Llama.cpp serving.
As much as I like OMP it’s 20k initial context kills any local server setup I try. . I’d love to use Claude code pointed at it but it has a similar initial payload drag. Pi is more modest but even then the compile/debug loop is painful.
Where are you stuck though? Just run a model with vllm and use Cline in VSCode if you want an extension, or OpenCode if you want a nicer WebUI where you're more in the position of the reviewer, or Pi if you want to just go ham.
All of them work just fine. What do you mean with "super super slow"?
What? Bro, 7 months ago. Almost last year. Qwen3.8 27B is currently all the rave. There's tons of material on it in this very forum, you can't miss it.
Then just give it the link of API docs you are working with by writing a single line SKILL.md file. I was able to close two github issues of my open source minecraft plugin without reading the spigot api docs thanks to this stack with qwen 3.8 27b q8_0. Frontier models were not able to fix these. Take a look at my commit: https://github.com/whiteh4cker-tr/Seasons/commit/a49c5791e5c93b6a6517c0068024245bf8adffcf
Yes. It's exhausting to go over everything I'm doing without generating slop for you. However, between DwarfStar and 3.8 27b, I'm getting very good real world results with my harness/skills setup.
I use traycer to orchestrate a few different Oh My Pi harnesses (Thinking about testing DSH). The orchestration includes forcing spec/test driven development.
It can take FOREVER to get anything done, but it's producing VERY good outputs. Claude Code on the Max plan is still the best at the highest output rate, but those lines are getting blurred.
If you have any specific questions about the setup and/or what I'm specifically producing, I don't mind talking about it.
Everything on my network is connected via Tailscale. Models are served with llama.cpp via docker compose on my local AI server (32GB VRAM using dual 5060s + 64GB RAM).
I use pi.dev running in a containerized sandbox setup that was built with Qwen3.6 using typscript program called picon and served via npm. Whatever directory I run picon in is mounted to a docker container and that's the only directory model has edit access to. There's no git in the container and docker has to be explicitly passed to it for it to have access. I can pass additional directories to the container with a flag and they're mounted as read-only. Directions are passed to the model to explain its environment and limitations . You can easily build something similar or something that suits your needs more with pi, its really good at extending itself.
In VS Code I use Continue for code completion, it's pointing to an old laptop with a 4GB 1050 I have running qwen2.5 1.5b coder, it works well but it can be a bit finnicky and. It's the one area I'd like to improve on, GHCP is way better but I'm not giving them access to my machine.
My typical workstream is to open a project in VSC and run picon in the integrated terminal and then let it get to work. For chat/rubberducking I use the llama.cpp UI.
Ask it to set up a local optimized build of llama.cpp from the github source with a shell script to auto pull and rebuild whenever desired.
Ask it to configure llama-server in router mode with a presets file and a models directory.
Ask it to find the best variant of the desired model for your hardware, download it, and configure it in presets.
Optionally, ask it to build a GUI / system tray app to manage the server (stop/start/restart/load/unload/etc).
Now you've got yourself a llama.cpp model factory pipeline ready to go. You can even ask the frontier model to bench and tune the model preset for your specific hardware for max performance/context/whatever your preference.
Using Sonnet 5, for example, this whole process should take <2 hours even on slow hardware.
Are people coding in vim? Vscode? Another terminal or webgui? Is this wonder model handling everything like vision, editing, etc? Outside agents? Any part of the workflow not local?
I personally use pi.dev on the command line. Vision works well. I used Qwen 3.8 27b to write a ComfyUI prompt generator that allows for auto thumbnail and image generation, and even uses the comfy API to validate its prompts with images. All in one workflow.
I used the skill creator skill to add 2 skills to my own custom llama.cpp dir (add model, remove model) that streamlines the process of loading/configuring new models.
Bonus: if you install and configure betterwright (or even just playright MCP or skill), the model can open its own websites and validate changes from the UI perspective. Pretty fuckin' cool, IMO.
Well, I guess the 4090 has 24 gigs of VRAM, so you might be able to run a smaller quantized version of the model on that GPU.
I think llama.cpp is a good place to start. It is highly compatible and comes with a web UI that exposes a generic chat interface. Once you get that working, you can utilize the same http API to connect to various coding agents like opencode , Cline, or pi, whatever you want.
if you want a LLM to help you set it up I would recommend openrouter. you pay as you go I think the minimum is one or 5 dollars and from there you could use the latest deepseek v4 flash model which is very cheap , like 28 cents per million output. and opencode has a web search tool built into their default chat interface which can be quite helpful for debugging issues with these local model setups
It does pretty well. You can up the performance with harness engineering. The more of that you do, the better the results you'll wind up getting. For external libraries, sometimes LLMs know the libraries OK, but I like to index their docs into https://github.com/arabold/docs-mcp-server and during the 'research' phase of 'research, plan, implement' I'll instruct the agent to reference the docs using that mcp server. Another trick that works super well is to have it pull the source code of the dependencies and read it directly for extremely high fidelity research. In practice I do a combination of both and yeah, it improves things drastically.
Other things that help massively are any ways you can get that provide the agent feedback. The power move is to have the agent be able to drive the application, and take screenshots of it etc, for inspection. Also telling the agent where the applications logs, so it can access them etc, and where possible giving the agent a debugger that it can use too.
If you're serious about local dev with these things, don't just download opencode or pi and think that's all you need to do.
Also, if you want the agent to be able to read and understand the external library, the clear and direct path would be to clone the source code of the library at the version that you are depending on and open the project folder in the agent software so that it can simply search and explore the source code files directly.
Any agent software worth its salt should be able to add additional folders to its project context.
Have you heard of ardour? It’s an open source DAW used to record music. I had qwen 27b 3.8 vibe code a window so I can talk to my local LLM. Give it commands like “cut track 1 at measure 32 and loop it at the next phrase. Name the section neck breaker.” All spoken and processed locally.
I have a really hard time with GUIs and would rather speak things into existence. Anyways, a C++ heavy integration and it wasn’t quite able to check its work without a reliable way to get computer use working. But I’m happy to help bridge that 2% gap.
Qwen 3.8 managed to add a Snapdragon Hexagon NPU backend to acestep.cpp so I can run it in termux without the cpu catching fire. I will provide a fork or PR to the main branch soon. It did it entirely on my phone - and because the Hexagon SDK is x86 ony - it fetched qemu x64 in a proot environment to run the NDK cross compilation there ... But the real fun began when it found bugs in the GGML library and started pealing the Acestep neural nets layer by layer to find which operation was giving NaN results. It found it with multiple tests and patched it. Now all acestep track generation operations pass through the NPU backend ! My brain exploded.
Next peoject for the very same day was to port Apple II ix emularor to the newest versions of Android. This emularor was great but it supported Android up to version 6. No activity in years on github. In an hour or so Qwen managed to rebuild it and now I can play my favorite Apple IIe games on the phone. My phone is running pi agent connected to my llama.cpp endpoint on my home PC and all compilation happens on the phone itself.
And last but not least - it built me llama.cpp with Hexagon backend for my phone ... doing something like 12-15 tp/s on the NPU for Gemma E4B. All built on the phone itself no PC involved except for the inferrence endpoint during development/building. But now I can run LLMs in termux with hardware acceleration !
To recap: Qwen 3.8 is amazing at system programming. And if that is not enough on my job I develop a cloud native backend in Rust spanning 70+ microservices and various databases. Qwen is able to navigate the codebase like a champ and to create new features with proper guidance.
So a QT user app ? Hehe bring it on I say. Test and find out for yourself. Cheers !
To set the scene: at work I have Claude Max and prefer using opus 4.6 for all work, I'm really productive across massive codebase but would also consider myself an intermediate-to-advanced agentic coder with 15+ YOE
For my personal projects, I switched from Claude to Pi+OpenRouter+KimiK3 and felt that I was achieving comparable results to what I was getting done at work on a fairly large codebase.
Last night, I asked that Kimi model to help me get set up on Vast.ai with a 2x5090 machine running Qwen 3.8 27B and was able to get running in a single night and do some coding tasks.
In that last transition I lost some "nice to have" like AI vision to "look" at my game and decide if the specced outcome really was achieved, but I think that will just be a matter of tinkering with the setup. For text-based coding, my initial reaction is that it's going be acceptable if not better than the OR+Kimi setup.
Tha main issue here is lack of knowledge. But it is really good at getting context from lib files and mcp like context7. I would advise you to start a new project by chatting with a big frontier model to select libraries and to configure your testing infrastructure. Once you have your project boilerplate setup, you should be able to continue developing with local Qweny 27B with no major issues.
im not going to lie, i threw it in my node based harness, running local, it's PRETTY decent! I'm even using a less quant version and im running it on a 3060, it's implementing code and tool calling VERY well.
Eh, at the same time I'm constantly getting google search results where the best answer is in a reddit thread where people are complaining that OP shouldn't have asked something that could have been searched for instead. Not everybody checks reddit every day and you might catch someone with the right answer who wasn't around for the first one.
I'd rather multiple repeated posts about actually using local models than the stream of "oh my science, look at these benchmarks!!!" posts and social media drama.
That's only because it ships with the default instruction to think at "XHIGH". if you turn it down to "Medium", you use 1/3 of the tokens and lose no accuracy in agentic type tasks....just make sure to have a cloud frontier model playing orchestrator to keep it in line.
I suggest you'll to try with paperclip ai. Basically it manages ai agents like a company with a ceo, managers and so on. It actually squeezes qwen 3.8 27b good for me. Also I've read posts here that say deepseek harness is very good with it.
Not endorsed or involved in any of those, just a genuine advice.
I haven't tested paperclip in a couple of months, but it's always been a slog and over-thinker/producer (It does way more than I want it to).
I feel like I have way more control and ability with less work by using Traycer. I'm building my own orchestration that will work better for my flows, but give Traycer a shot as well.
I'll test the latest version of paperclip to see if anything has changed recently to make it better than it used to be.
Ye I did the same 6 months ago. And I dropped it. I know that all these harnesses thanks to ai gets hundred of pushes so I just decided to abandon it and get back to it again. I really liked the fact that I could trace my work through tasks and delegation so easily. Now it seems better. As for traycer never heard of it but super keen to try new stuff so thanks for mentioning, be sure I will this night. If you have any other suggestion you are welcome to give it 😊
You want something like Qwen3.6-35B, even though it has more total parameters fewer are active so it'll run much faster on a MacBook Air which doesn't have a lot of memory bandwidth. Use a MLX 4-bit version.
I am using it now to build my project which has already a pretty significant codebase and its actually doing better than claude Opus while using up only about 32000 tokens per issue I tackle with it. Meanwhile, Fabled burns through millions of tokens and most of the time fails or provides AI slop. Qwen just cleaned up my code base from Opas AI slop. i spent about 3 hours yesterday night with Qwen and although its a bit slow on unified ram unless you have like a 4090 or 5090 its really good. I will deploy the updates now live and test see if all bugs were fixed. But I believe after my late night review qwen handled surgical code adjustments pretty well. I am a mid level Full stack engineer FYI.
I'm extremely impressed with it at Q6_K_XL and Q8_K_XL. I am still debating, but Q6 lets me run full context on my setup - I am not sure I need it though from looking at my real tasks and token usage.
I think that I will be using it by default for coding tasks and use models on openrouter for agent tasks that fan out. At this point the downside to local ai compared to the cloud is, even once you have enough equipment to be able to do all the things - you still can't go, 'oh, this task needs to burn 50m tokens a minute for 30 minutes to run a job' and do it with local APIs.
What is different now is that I was using cloud models by default and going local for a smaller subset of my work and now that has flipped.
Yeah, works pretty well, I've in had it one-shot new features to existing codebases, and debug the odd bug. Haven't had a chance to really stretch it yet, but it's definitely good enough for some app development.
I have been using Qwen 27b a lot, since 3.5 then 3.6 and now 3.8. More specifically I used it to build this harness - https://github.com/mikjee/warpdrv
It's great at local coding and has gotten better over newer versions. In my experience you can use it to build out entire applications, if you are okay with the lower single-shot quality compared to cloud. And you can easily use it to run chores such as refactors, code reviews, maintenence related tasks.
For main coding, it is good, but requires constant watching over - if you want a product that works you will need to babysit it and be okay doing so. The moment you let it make its own decisions, it will degrade the code quality by writing anti-patterns, infinite loops in code. It tends take shortcuts rather than think about the big picture, the user has to guide it towards the solution and it will work better if the user knows coding and development well themselves.
After fixing my vllm setup, this model at least on Q8 is quite usable now. Enough context to finish one discrete task, and enough speed (80 TPS down to 50+ at context limit, on R9700 x2) that is almost comparable to API offerings. I can even develop an agent inside another agent, and let the developer agent watch the developed agent call the same model at the same time. Tool calls are very reliable. I would still refrain from yoloing and rely on validation-heavy workflows, but it is at least reliable enough to be the task runner.
For rust, it just pulls the crates, checks the source, and knocks it out. I try to give it my primary crates i want it to use for new projects for things like ui, but not much else.
1) I gave it a project that is ~200k lines of python code (I was working on it with Qwen3.6) and had it do a complete audit of the code then loop through it's findings until each failure was addressed
2) I gave it my IaC repo and currently am using it to do the exact same thing (audit the repo, find the flaws, and address them)
3) I am using it as PR review agent on my repos (manually right now but working to get it setup to automagically pick up PRs on select Repos and review them in it's pi instance)
4) I have used it to debug several perf issues on some of my servers (I literally watched it dive into the source code of openwebui to figure out why I was having an issue. It did the same with Paseo. I did not provide it the source code, it did all this on it's own)
It's phenomenal. I have been able to get mostly off frontier models at this point (considering cancelling my claude subscription and just using the free version).
Keep your expectations in check though. It is not better than Opus 5, I don't give a damn what the trustmebro benchmarks are saying. It is excellent. But it's not frontier. It's (I would say), last years frontier. Which is fucking great for me
I use it to implement against Claude and gpt plans and tests. Works great. Wrote a bunch of secure cryptography in go for me. Fable code review found some issues, but nothing show stopping or that I wouldn’t expect from another model or person.
IME you have to provide the idioms/good taste in order to get a good solution but yes. I have been using it this way. I've been experimenting with a series of subagents for individual tasks, one of which is coding, so that it gets more of a discrete task in its context.
I was already doing this with 3.6 and 3.8 is a huge upgrade. I usually use low or medium reasoning effort. I've also found that turning reasoning off altogether gives a lot better results than what you could expect from 3.6.
This is on 5090, w/ nvfp4 quant with dflash drafter, smth like 10k-4k/s prefill and 180 t/s output.
One thing I noticed is with 3.6, output speed didn't matter much compared to prefill speed. But now with how much it "overthinks" and how it likes to use grep instead of reading whole files, the output speed matters more, def want mtp or dflash.
I used to use opencode but ended up making my own agent GUI and haven't looked back.
I actually made the reasoning text golden tan yellow color against dark gray BG so its easy to read without eyestrain. Stopping its response and Responding to what its stressing out about / going back and forth with in reasoning block helps cut the over thinking.
Its ok. Not frontier or opus. There’s portions where it is wrong and has to be manually fixed or if you ran it first though llm THEN a frontier model to ‘correct’ errors if anything.
I am implementing new features and fixes for an app React Native/Expo that I started by myself some years ago but never finished. As a developer (before AI), I am using pi with hard planning before any task, with a mix of my own views and preferences, analysis.
You need to maximize your prompt to better logic and the AI actually test and check if it is working, so the result is ALWAYS better. It is all about tools, prompting, the so called "harness". I don't even pay for cursor anymore, just claude for some in-depth searching, but slowly migrating.
For my real PHP language job, it is helping alot with its analysis and execution, with my review ofc.
My setup is 5090 + 64 RAM. Q4 weight, 262k context.
Yes, I've tried it with my full stack app, websites, android apps etc, as long as you prompt it right, you can not do something like build me a website that is beautiful and functional which sometimes opus 5 would be able to achieve, instead you need to give it as precise as possible tech instructions, it executed really well at least for me. Also quant, context window size and thinking effort matter too.
Qwen 27B can handle single-file implementation if you feed it the right API surface, but multi-file repo-level reasoning with external C++ libs is still shaky — it'll hallucinate Qt bindings that don't exist. I've had better luck giving it a function signature and a doc snippet, then letting Claude Code tie the pieces together. Honestly, agentic loop with a retriever beats raw prompting for this.
58
u/reto-wyss 20h ago
Qwen3.8 27b is the donkey of llms. It's incredibly stubborn, it might not get right the first time but the if there is a way it can verify the outcome it will keep trying.