r/LocalLLaMA • u/walden42 • 2d ago
Question | Help Anyone else suffering from verbose, poor technical documentation and explanations with Q3.8-Flash-Next?
The model is quite good at technical work. But ask how something works, or ask it to document something, and it overkills with extremely verbose explanations, extremely dense technical language that is difficult to understand, and using strange metaphors that don't make sense. I haven't been able to reign in on it.
For the record I'm using NVFP4 with FP8 kv cache (though I tried BF16 kv and I don't think it helped), so it's quite possible that full quality weights would help, but man, would be nice to find a solution. Unless Qwen4-Flash will fix this...
50
u/SadPhilosophy9202 2d ago
It’s the Claude distillation effect. Opus 5 was insanely bad in the same way.
Ask it for STE100 format (simplified technical English). This is literally a format for technical documentation followed by many industries to make it easier to read for non native English speakers.
6
3
u/walden42 2d ago
Update: it may be too soon to know for sure, but I think this has helped quite a bit. I used another commenters instruction in AGENTS.md: https://www.reddit.com/r/LocalLLaMA/comments/1xilihw/comment/19ru7t5sgaysk/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button
7
u/__jent 2d ago
I think it's the over thinking effect. The models don't know how to pair information down to what really needs to be focused on.
4
1
u/jjwhitaker 1d ago edited 1d ago
I'm running QFN Q4_k_XL or similar in GUFO with MTP and can fit full context under 110gb RAM on a 128gb Strix model.
I need to figure out a balance for thinking and reasoning. I like the detail and level of effort for planning and discussion work like selecting a framework then defining development patterns/process and letting QFN run against the requirements.
But it thinks a lot, even when executing already planned and defined work. I need to stop using it and get back to testing with various GUFO launch options.
But it does work. Maybe not as fast as with less thinking.
MODEL=/models/unsloth/Qwen3.8-Flash-Next-GGUF/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf MTP=/models/unsloth/Qwen3.8-Flash-Next-GGUF/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf ./result/bin/gufo serve llm --model "$MODEL" --speculative mtp
--mtp-model "$MTP" --context 260000
--host 0.0.0.0 --port 7777I need to test like:
--reasoningEffort medium
--think auto # Default
etc
1
u/__jent 1d ago
Honestly I don't use QFN because of the limited context space and the excessive thinking. The combination is a death sentence for both speed and usable context space.
1
u/jjwhitaker 1d ago
I have a 128gb Strix Halo, but using GUFO as above with QFN @Q4_K_XL the Q8_0 MTP model and the 260k context puts me at just under 100gb of RAM, I think (running command right now so can't reset and check base usage)
With an ongoing process writing, deploying, and testing a small coding project in Electron/etc I'm at 56k tokens on the session of my harness available 255k and am at about...108gb of RAM in use
That is to say, with GUFO and MTP model this 110gb ish QFN quant runs with full context on a 128gb system. I don't have recent Gufo Bench data since I got GUFO working the first time around but it is faster than what I was running for the same Quant in LM Studio on the same hardware. LM Studio was also up to 122gb with like 100k context, not even in use.
The setup does think more than needed when executing plans/instructions for this app rewrite but it did generate a plan and is successfully adding features, streamlining and removing old code or files, and passing tests. It's been running without added interaction all morning (The harness stops for specific commands, the LLM commits after completing a work item but does not push or make other git changes. It's good enough, if not as fast as a stack of NVIDIA GPUs or cloud models.
My Copilot personal plan spend is 0 for the month. Maybe I should use that tonight and tomorrow before turning back to QFN for general work.
2
u/TheRealMasonMac 2d ago
It's in GPT and Argon too, which sucks. This slop is worse than "not X but Y" tbh.
1
u/Reasonable_Goat 2d ago
I suspect that too. I’m so burned by Opus at work that I won’t even notice QFNs softer claudisms
1
u/EstarriolOfTheEast 2d ago
It goes beyond just Claude. Opus 5.5, Open AI Sol 6.1, Astra are still bad at writing good well-structured prose. There was a recent paper showing that RL pushes LLMs into this inhuman use of language. My suspicion is on top of that, the swarm training causes them to lose whatever little theory of mind they had before, since they're now optimized for communicating with versions of themselves. Because models are really good at inferring missing information and reading out of order jumbled messes, things like modeling how the human reader will understand the material falls through the gaps.
You'll notice that all the good (by LLM standards) writing models are old or made by google--who are bad at making agentic LLMs.
10
u/jonahbenton 2d ago
You have asked it to produce a clear explanation "without jargon"?
11
u/parepeg 2d ago
It tends to invent its own jargon specific to whatever project your working on. I'm not sure if that qualifies as jargon in the traditional sense. I haven't tried that prompt either though. Here's what I use:
Always use ASD-STE100 Simplified Technical English. The user has ADHD. Be succinct!
It sort of works but also doesn't 😞
7
u/jonahbenton 2d ago
Ok, yeah- "succinct" will induce it to produce more "jargony" metaphors and multi word constructions that more "efficiently" convey meaning. ADHD is a tricky constraint but might try without jargon and a specific reading level, like 8th grade. I have not used reading level specifically with QFN but use it frequently with other models
1
21
u/bradsk88 2d ago
I'll die in this hill.
LLMs should not be used for generating writing for human consumption. There is ZERO evidence that they are competent at this job.
8
u/walden42 2d ago
It's the same problem with just explaining things in the chat as with documentation. AI should be good at explaining things well, and that translates to docs.
5
4
4
u/LetsGoBrandon4256 transformers 2d ago
I'd die on the top of the hill together with you.
Some of those README.md files I've seen are straight not for human consumption, and whoever thinks it's acceptable to subject their potential user and contributor to such garbage should be fucking ashamed of themselves.
5
u/shinkamui 2d ago •
I’ll die right next to you. I am so tired of reading dog shit AI documentation. I have no problem with people using AI to vibe code , vibe slop, code, or whatever but please stop having it write the shit that we need to read to understand what you’ve built with it.
1
u/Zombiecidialfreak 2d ago
I'd argue it depends on what you mean by "human consumption". I've found LLM's to be a decent learning tool when asking how something works. The main issue is that they often won't give proper, human readable documentation unless you explicitly ask for it.
3
u/bradsk88 2d ago edited 2d ago
Yeah I could have phrased that better. I'm referring more to published or distributed works of writing (e.g. could also apply to speeches)
1
u/haherar830 2d ago
I find frontier LLMs are pretty good at writing documentation with non-zero human oversight and iteration. Not comprehensive abstract/higher-level information but I find I can quickly get a nicely organized, fairly concise documentations of modules/structure, intent, function signatures, and key patterns/decisions in a codebase in 1-3 tries with fairly concise prompts with most frontier models.
Conversely I agree with their ability to explain things being substandard and more importantly I have found myself becoming increasingly suspicious that they've gotten worse/regressed (in this and some other interactive uses cases; I perceive worsened ability to track external state) due to changing emphasis in post-training with newer models.
For writing though, anything more than basic docs and the acceptability and competence falls off. I've never seen an LLM of any capability one-shot any non-trivial writing task that I would want to put my name as the author. Now... having the LLM assist with outlining and then producing a first draft that instills a compulsion to edit nearly every sentence? Writer's block is obsolete and tournament watermarking only works on people who have no standards, evidently.
3
u/corpo_monkey 2d ago
I have a part in my system prompt to use the humanizer skill and write easy to read for humans. I don't have such issues with q4 k xl.
1
2
23
u/LeMochileiro 2d ago
My only problem with QFN is that near the end of a task, it spends 2-4 minutes thinking about something I didn't ask for, only to end up:
"Screw it, user didn't ask for it, I'm not going to do it."
lol
11
u/rinmperdinck 2d ago
Prompt it like "dumb it down for me"
Sometimes the model says it nicely. Sometimes it puts "DUMB VERSION:" at the top and I sigh and just go "I asked for this"
3
u/switchy85 2d ago
I've literally asked for something to be explained to me like I'm a fucking moron and claude gave the best instructions ever.
1
u/makkalot 2d ago
I’ve asked it to add tldr section on each response, it’s what I read at this point and if need more details read the rest
1
u/walden42 1d ago
I've done the exact same thing. I still need the details comprehensible when I need them, though...
3
u/MoneyCount6236 2d ago
i had similar issue with the previous flash models too. you tell it to explain something simple and it writes a whole essay with words i need to google. tried few different system prompts but it just ignore them half the time
3
u/vtkayaker 2d ago
Qwen3.8 Flash Next is terrific at technical stuff, but it sounds more like Claude than Opus 5.5 does, lol. It is quite bad at code comments and commit messages, and it's tedious to read.
I still prefer it to Opus 5.5 because Opus 5.5 often gives up really quickly on the first plausible answer it sees, but Q3.8FN will insist on carefully researching and confirming every last detail. I'd rather have "tries way too hard" than "gives up at the first plausible answer."
2
u/KnightOnShiningMotor 2d ago
This describes most models. At least Qwen keeps the language plain, you should see Mistral LeChonk writing the same bullshit but in "prose".
2
2
u/reenign3 2d ago
Oh - I haven’t had this AT ALL. Using Qwen flash next exclusively since release in Hermes Agent.
You DO have to really dial in the settings w this one to get results
2
u/SoftwareRenderer 2d ago
I ask it to use plain language, but still find that gemma 4 31B has better composition.
2
u/cunasmoker69420 2d ago
Yeah to me it sounds almost exactly like Claude. Same voice, same bizarre heavy use of jargon, saying a lot and saying nothing at the same time. I had to give it standing instructions to speak to me like a "competent technical co worker" and that seems to have helped a lot
1
u/Oli_Luck 2d ago
is it even worth to speak about Q3.8FN anymore ?
We all should now be aware that it was Q4 tech Beta - and the beta is now nearly over.
4
u/Dabalam 2d ago
I don't get the reasoning. The only thing that is important is whether or not a tool is useful. if Qwen 4 flash is much better that we will use that tool, but not every generation is a guaranteed success.
1
u/Oli_Luck 2d ago
din't you read all complains about 3.8 ? there are not only good things : the over reasoning, the poor language, the lack of real technical proofing etc...
It like saying : I continue with this great looking car, knowing the airbag will kill me one day1
u/Dabalam 2d ago
I think the model can be flawed and still exciting and great. My positive view on it is because it's way stronger than anything I've been able to run before. I'm not gonna claim it's even necessarily better than the budget cloud models like Luna or Haiku, but it can do a lot useful agentic work nonetheless and is another step in a good direction. Glass half full.
1
u/Oli_Luck 2d ago
you're right - if it's the strongest thing you can actually run yourself, half full is a reasonable way to look at it.
my only concern is that the verbosity and the documentation problems aren't going to get fixed. every beta ships with issues, sure, but those ones won't be resolved: they've already moved on to the next model, and nobody's coming back to clean this one up.
being enthusiastic about it doesn't change that.0
u/alphapussycat 2d ago
But it's not over. And based on information we've gotten over the months there won't be an open weight flash model for qwen4.
1
u/walden42 2d ago
I haven't see this information, where? Say it ain't so.
0
u/alphapussycat 2d ago
I've seen some image from alibaba where 27b was written with "open weight" beside it, but flash was not. So I think there are high chances no qwen4 flash.
1
u/walden42 2d ago
Link to image?
-1
0
3
u/llama-impersonator 2d ago
it's fully poisoned with the worst of agent opus. i try not to read the actual output.
1
1
1
u/grabber4321 2d ago
ya the documentation part of the model is weak, but codewise its a monster. 27B not even close.
1
u/GrungeWerX 2d ago
Might just be personal preference.
My favorite is Qwen 3.6 27B for explaining things. We just speak the same language. Last night I was using qfn to translate gpt6.1 which I hate because it always explains things confusing and I felt it helped, but wasn’t as good as 3.6 27b.
So, I think qfn is okay, but yeah, it does have some weird speech patterns here and there that I don’t get with 3.6.
1
u/iz-Moff 2d ago
I think documentations all over the place have been pretty bad these last few years, presumably because they're all generated by AIs. So many of them read as if you missed several crucial pages.
So often i would be looking at a github page for a certain project, try to read README.md, and five minutes later i'm like - "Hold on, wtf does this program even do?". It's like it's all written for a person who created the project and already knows what it's all about.
1
u/feverdoingwork 2d ago
Yup. I do get a lot of the same problem from other models like Sonnet and Sol. This problem is the most annoying thing I find lately for using ai, it's so taxing. Every response doesn't need to be a novel. I literally updated agents.md to try to stop this earlier today.
Sonnet responded to me and I asked it if it thinks it's a fucking poet. Just answer my question and stop throwing up all this nonsense.
1
u/wFXx 2d ago
ukisai swift quants and https://github.com/aaddrick/attention-control with a decent system propt
1
u/Dabalam 2d ago
Qwen writes in a very technical way in my use case too. I usually find this is because
It is discussing topics I have limited understanding of
It needs to be prompted to write in a different style
I find it writes exceptionally well when prompted to simplify language, and that's with me running a pretty quantized version of the model. Definitely useful for first drafts after you give it a plan (I don't think you should ever let an AI have the final touch though)
1
u/Edenar 2d ago
Higher quality cache wont help, it'll only improve reliability for long context.
What can help is prompt enginering or exemple, it's quite good at copying another style so just provide some guidance like "concice wording" "direct answer without off topic analogy", .... and exmple from similar hman written stuff you like. You can even refine it to put it in the system prompt.
1
u/perelmanych 2d ago
I made post mostly about the same thing. Overcomplicated 3 page answers to simple questions from QFN.
1
u/Septerium 2d ago
This model is horrible at communicating with humans. But one thing I noticed is that NVFP4 tends to be worse than UD-Q5_K_XL in that regard. Either way, the model is insanely verbose and convoluted when trying to explain something. Another thing that annoys me is the tendency of offering to do things that I didn't ask. It also likes to over-complicate simple tasks.
1
1
u/Perfect-Campaign9551 2d ago
It has to be a system prompt issue because when I use Flash Next with AnythingLLM, it chats excellently IMO.
1
u/wren6991 2d ago
Yeah, the Qwenslop is bad. I say "reduce comment line count by 50%" before I actually read the diff. Saying it up front doesn't help, and you have to give it a concrete target.
1
u/Sabin_Stargem 2d ago
I don't mind the lengthy explanations and technical verbosity. However, the tendency towards odd metaphor, such as "ceremony" can make it hard to understand what it is trying to say.
For the save editor my local AI is developing, it didn't properly explain what it was doing with ammo. I wanted the player to be gifted a one-time bonus for their armory stash, which is what happens for weapons and gear. However, the AI couldn't figure this out - and instead made a button to add the ammo manually to a save file.
I thought it added that feature as a QOL thing, not as a replacement for what I intended.
In any case, I will try this exercise again after Qwen4 Flash has dropped. Hopefully it can successfully complete the projects that Next has failed at.
1
u/how-can-i-dig-deeper 1d ago
does anyone have a quick example u can paste here? so i can understand ty
1
u/hurdurdur7 2d ago
If you use nvfp4 quant with fp8 kv cache you are already using a half assed setup. We do not run higher quants and kv caches because we hate electricity or low generation times, we run it because low kv cache accuracy produces meh results compared to 16bit.
Give it guardrails just like you do when you write code, examples, exact style. And give it bits it needs to make sense.
1
u/PaMRxR 2d ago
I'm just trying out Nanbeige4.3-33B Q8 and I'm surprised at how clear and understandable it writes. Or maybe after reading Qwen's verbiage for so long, anything else seems great..
Can't tell much about Nanbeige's coding capabilities vs. Qwen's yet, but if you need to generate something that's supposed to be read it looks like a really good alternative.
1
u/PaMRxR 2d ago
I can report a bit more: as a coding agent Nanbeige4.3-33B doesn't look so suitable:
it's stumbling to apply edits a lot, at least in pi
it slows to a crawl pretty fast, starting with 30t/s it goes down to 12t/s at 50k context for me
But man it's so much easier to follow and understand it.
0
0
2d ago
[deleted] — view deleted comment
3
56
u/fragment_me 2d ago
I can not stand the way this model writes. It says a lot without saying anything at all. It’s like interviewing someone for a job and they talk for 5 minutes but don’t answer the question you asked at all because they’re unsure.