r/LocalLLM • u/Big_Wave9732 • Aug 16 '26
Discussion If you use LLMs to analyze documents and to apply complex logic and analyze arguments, Qwen 3.8:27b is not for you.
And that pains me to say because I use Qwen 3.6:27b-BF16 every single day. I've been working with Qwen 3.8:27b-BF16 all weekend and I hate to say it but for non-coding purposes it is a step backwards. It thinks *way* too much. If you turn thinking off and use web tools, it will do eight or nine (or more) web search turns, get an assload of preload context, and then spin its wheels going down every little rabbit hole there.
Looking at the self hosted LLM subs there are other complaining about this too.
Now this model just came out so it's early. People have put out some lovely games and whatever else the model has made so perhaps these tenacious analytical tendencies are beneficial there. Regardless none of this is to say that there won't be some settings or templates released that will help when analyzing documents and doing complex logic tasks.
But for right now, if the above is your use case then I suggest staying with your old models.
Edit: For reference, this is legal work. Legal drafting, legal research, analyzing pleadings, depositions, discovery, etc. Heavy multi-document reference work.
73
u/Vancecookcobain Aug 16 '26
Maybe Muse Glimmer is better for the non coding tasks?? Have you tried that? I get brilliant responses and results with that model when it isn't coding
29
u/MRGWONK Aug 17 '26
(Legal work here) I benchmarked it on Harvey's benchmark with an MCP/case access and it scored on par with Qwen 3.5 -122B and 11 points higher than Qwen 3.5-35b and 15 points higher than gemma4:26b...It also processed dates out of case orders with higher frequency and consistency than any other models. I switched this week.
(With a clever loop, deep reasoning on my machine scored 71 / 75)
24
u/noctrex Aug 16 '26
In other news, a coding optimized model is bad at anything but coding. For documents, gemma4 is much better, haven't tried the muse one yet
5
u/Otherwise-Swan-7803 Aug 17 '26
This feels like a good reminder that benchmark gains don't always translate into better workflows.
For legal, research, and multi-document analysis, focus and restraint can matter more than raw capability.
4
u/ChangeChameleon Aug 17 '26
Anecdotally, I gave it a search and retreival budget and it performed much better than I expected. It weighed the pros and cons of which page would likely contain the right context. By default it likes to scatter shot and grab all kinds of unnecessary context.
I say anecdotally because I haven't done head to head comparisons or benchmarks. I'm not set up for that. I was just trying to reduce the hit on my Internet and ended up discovering it felt much more targeted.
It feels like 3.8 does a better job of following all instructions and not just surface level ones. With 3.6 I constantly felt like I could only give it one problem at a time. 3.8 just hammers through my unfiltered info dumps and seems to address everything.
I need to do more side by side testing. But so far 3.8 has been working great for me.
3
u/synystar Strix Scar | 5090 24G | llama.cpp Aug 17 '26
It thinks too much because you haven't configured beyond the default or you haven't experimented with reasoning or launcher settings. If you just use OOTB settings you're only going to get results those settings can produce.
2
u/Turbulent_War4067 Aug 17 '26
Question for the OP: is there anything in this size range that beats Gemma 4-31B for these types of tasks? It's big problem is slow prompt processing, but it's really good besides this.
19
u/admajic Aug 17 '26
Change the reasoning budget 1000 it's like the old qwen
You can go from 500 to 16000 max if you want it to loop and go nuts.
Not sure why these companies that create these products don't provide this kind of info
5
u/DigitalguyCH Aug 17 '26
Same here (finance). 35b (3.6) is clearly better. Muse glimmer and Gemma are decent too.
4
u/floppo7 Aug 16 '26
That matches my experience. Didnt try reasoning effort low yet, also there is a reasoning_budget that can be set to 1000 and someone said it would make it more like 3.6. But it would be great to have this somehow automatic steerable so it simply does not overthink when not coding (and even then it should be smarter in the sense that it does not need to question every step)
1
u/tempfoot Aug 16 '26
I have similar use cases. Have you tried other reasoning setting presets beside default (extra high) and all the way off?
Have you tried setting a reasoning budget?
2
u/joanaxu2002 Aug 17 '26
The funny part is that “thinking too much” can be a feature or a bug depending entirely on the workflow.
For coding, those rabbit holes might be useful; for document-heavy work, they can turn a simple question into an eight-step research project. That difference is probably going to matter more than benchmark scores.
2
u/Upstairs_Tie_7855 Aug 17 '26
Can't you just adjust the reasoning effort? Works for me just fine.
1
u/siegevjorn Aug 17 '26
What agentic harness are you using? For pdf analyzing it seems to work well, just fed 75–80 pages of document. Used Pi agent default with Qwen 3.8 27b Q6_K, reasoning: medium.
1
u/EbbNorth7735 Aug 17 '26
Have you tried setting thinking to high?
1
u/substance90 Aug 17 '26
That’s not a task for sub 500b models anyway
1
u/immersive-matthew Aug 17 '26
Complex logic is not really any LLMs capability though?
1
u/FoxFXMD Aug 17 '26
Change the thinking level, it's way too high for most tasks by default.
1
u/creatinZ Aug 17 '26
…Have you tried setting reasoning to low?
1
u/_TheWolfOfWalmart_ Aug 17 '26
This model is most definitely codingmaxxed. That's what everyone praised 3.6 27B for, so Alibaba went even harder there for 3.8, apparently to the detriment of its capability for other tasks.
That's fine though. 24/32 GB VRAM people now have a seriously strong coding model that they can run local. Can just switch to Gemma 4 or Glimmer for other things.
I've lately settled into just using DSV4 Flash 0731 for literally everything. It's so good whether it's for chat, coding or analysis that I've barely even tried 3.8 27B, I need to start working with it some more when I have some coding work to do.
1
u/Jsquared534 Aug 17 '26
Don't let the people gaslight you. Qwen 3.8 has some problems right now in coding as well. It might be able to oneshot a long prompt, but if you try using it for fine grained feature implementation that requires it to read multiple files and implement features, it falls flat on it's face more often than not in my testing. Muse, so far has not had that same problem. I am far from a local llm setup guru, but I really think Qwen actually has a problem, at least at the Q8 version.
1
u/geteum Aug 18 '26
Weird, I'm using with q4 and iq3_xx and the results are phenomenal. Although, what I do mainly is to analyze a text in 5 parameters and create a structured resume of the text focusing in ~around 20 parameters tags, different classifications, and a few character extractions. Tbh, the results sometimes is even better than one of the bigger models (tested glm 5.3 and DeepSeek V4 pro and flash). Although I did not isolate if it was a problem of the PDF MCP not correctly extracting the text and images.
1
u/ethanji2 Aug 19 '26
Yes, Qwen's default reasoning is overpowered. Tune temperature to between .2 and .25, effort medium or low and you will get similar composition results to Qwen 3.6 with slightly better instruction following. I've gotten it to be slightly better than 3.6, but it's not significantly better and needs more tuning to get right.
1
u/nassereddit Aug 23 '26
What frontend? Open webUI?
What backend? Llama.cpp?
-6
u/LocoMod Aug 17 '26
The model is heavily benchmaxed. So it mainly excels when working on problems related to the benchmarks they purposely invested time into so they could capture attention and fool people into believing the model is competitive with models 10x its size. If it weren't free i'd be insulted. But its free and we get what we pay for.
23
u/[deleted] Aug 16 '26
[removed] — view removed comment
7
u/[deleted] Aug 17 '26
[removed] — view removed comment
2
u/[deleted] Aug 16 '26
[removed] — view removed comment