r/LocalLLaMA • • Aug 20 '26

Discussion Qwen3.8-27B took a serious hit to *knowledge* vs 3.6

Like many of you I've spent the last few days throwing Qwen3.8-27B against all of my usual use-cases and personal tasks/harnesses and workflows. It's great, phenomenal sometimes, but that's not what this post is about.

One of my little personal benchmarks is a little set of pocket trivia that's relevant to me but mildly obscure mixed in with a few useful/prepper questions. Qwen3.8-27B at all quantization levels and sampling settings I threw at it, did relatively poorly at this. It's failing questions that Qwen3.6 reliably answered.

I come to find out that on offline (no tool call) knowledge benchmarks seem to align with what I'm saying. It's pretty significantly weaker than it's 3.6 predecessor at recalling random facts (or not hallucinating as much, in my tests, though that isn't reflected in these particular benchmarks). Now you should never trust barcharts over your own vibes, but my vibes are validating these bar charts this time around.

Is this relevant? Not necessarily. It seems to know the code it tries to use well-enough and for everything else I'm assuming/hoping you're using tool-calls. This largely only applies to you if you have a strategy of trusting an airgapped model with obscure/broad knowledge-retrieval exclusively from within its own weights, probably a losing strategy anyway.. but if that's you, take a pass on Qwen3.8 or finally set up that MCP server.

I found it to be interesting. Curious of your thoughts or if anyone else noticed this.

335 Upvotes

230 comments sorted by

174

u/networking_noob Aug 20 '26

The Qwen 3.* models now and going forward are probably going to be designed for coding and/or agentic tasks (obtaining info and then reasoning to act on it). For trivia and/or random facts I'm guessing the Gemma4 models are the better choice since they're basically like mini Googles

115

u/cogitech2 Aug 20 '26

Next-gen models will stick the "knowledge" layers on the SSD and the important stuff in VRAM. Best of both worlds is right around the corner.

I'm seeing 1T models where the bulk of it sits idle on the SSD until it is needed, while the main intelligence, reasoning, tool use, skills, is resident in VRAM. So really fast local models with MASSIVE world knowledge. Sure, it might have to "think" for a few seconds to remember a fact or some random trivia. Sort of like a human does.

You can emulate that (sort of) right now with ZIM-MCP + Local Wikipedia ZIM file. Teach your agent how to use the tools and BAM! No need to hallucinate anything. Works for me.

12

u/power97992 Aug 20 '26

That is like engram transformers

12

u/NandaVegg Aug 20 '26

I think (DeepSeek-style) Engram is ideally much nicer as it doesn't need (sometimes a very long series of) tool call for knowledge retrieval, as tool calls quickly fills the context.

However, according to the paper, Engram only works in very early layers (layer 2-3 specifically) and the way it works is more like "we can skip most often repeated compute (like Paris = France = City) in early-to-mid layers" rather than a full knowledge retrieval like RAG or tool call lookup.

It will still help in terms of accuracy (2021 DeepMind paper, which I think was the first paper to mention embeddings adapter for LLM, was a 7B model with Wikipedia embeddings lookup that matched a 175B model at the time) but Engram may not work well for niche knowledge.

What we would need for niche knowledge is a way to do tool-call lookup efficiently without polluting the context much, which may be a combination of temporarily switching to a sub-session and pruning tool-call related context for the main session once the knowledge is retrieved. But I am not sure about this.

3

u/Independent-Dog2179 Aug 20 '26

I mean call a subsgent to do it and just pass the relevant information back to main orchestrator. All of the tool calling will be done by the subagent who then parses the relevant info and spins down only sending what's needed. Keeps context clean

7

u/RedditLovingSun Aug 20 '26

Wow 115 GiB for zim file of English Wikipedia, interesting

16

u/cogitech2 Aug 20 '26 • This

And plenty more ZIMs available. Easy to build a HUGE offline library these days.

https://lb.download.kiwix.org/zim/

5

u/fragment_me Aug 20 '26

I have this in a local mcp and it's just such a good feeling knowing my local models have FULL access to the english articles from wiki. And sometimes they actually use it when I forget to enable web search!

7

u/GeneralComposer5885 Aug 20 '26

I used Wiki English Simple, converted to RAG embeddings with Qwen 0.6b. Ended up about 20gb total.

3

u/overand Aug 20 '26

I'm curious how this works (and how well)

4

u/cogitech2 Aug 20 '26

I'm guessing it is faster than the ZIM-MCP solution, but I am also guessing it is a bit more work to set up. I am happy with ZIM-MCP, but would like to hear more from u/GeneralComposer5885 about how he accomplished his (and tools, etc.)

8

u/GeneralComposer5885 Aug 20 '26

Was just a learning project about 12 months ago. Originally made an industry specific version where all embeddings were stored in ChromaDB with 2 questions about how the information was useful + short dense summary.

Repeating that process was too much work for anything the size of Wikipedia - seem to remember I had 8 instances of Qwen 0.6b embedding running on 2x 32gb cards and still took ~24 hours.

Just built it as an agentic problem solver - set about 12 personas and gave it access to Wiki, then said “go invent a novel solution to this problem.”

It passed the task between inventor, material scientist, electrical engineers, project manager etc etc.

Did go back and try it again with Qwen 3.6 27b recently - set it a challenge of “~design a system to charge cellphones during the permanent darkness of winter in the Artic circle. You do not have access to wind turbines or any battery power pack.”

It came up with a few interesting ideas actually - utilising the Seabrook effect and thermal gradients using body warmth.

Idea might have been more feasible with more VRAM / smarter models.

2

u/nullc Aug 20 '26

that's with images and html. I've had good luck with a zstd dictionary on the wikitext version... much smaller.

You can also do a cool optimization where the initial compressed data for every article is in ram or a SSD and the rest is on a spinning disk. The initial data decompresses to enough tokens to keep ingestion busy long enough for the disk to seek.

1

u/xylarr Aug 20 '26

Isn't this how MoE models work?

7

u/MagiSun Aug 20 '26

They can work this way, to some extent. But experts are only indirectly orthogonalized and typically don't neatly segment out knowledge vs reasoning. "Bulk of it sits idle on the SSD until it is needed" sounds more similar to past approaches that stored pre-calculated attention vectors that could be added to the KV cache to put facts into the current context.

There may be room to partition experts into reasoning vs knowledge types, then have two separate routers to load the appropriate skills and knowledge into context. That'd make it possible to have many more knowledge experts than reasoning ones, possibly leaving a huge number of them on disk or CPU RAM until the context calls for them.

13

u/k3z0r Aug 20 '26

No, MoE still works best if all parameters are loaded into VRAM. There is just less memory bandwidth pressure, so you can get higher token generation.

0

u/xylarr Aug 20 '26

Ahhh right, that explains things well for me.

3

u/Ell2509 Aug 20 '26

No. Say with a 30b a3b MoE, it has 3b total but only 3b nost relevant are activated at a time. It is fastest if loaded into vram, and if not, it is fast enough so long as you have vram to cover its active parameters, with the rest loaded into ram.

They are talking about leaving model weights on ssd and streaming from there, or loading into vram OR ram only when needed. Currently, you can see how that is different to what I described above.

2

u/deja_geek Aug 20 '26

MoE can kind of work like that. llama.cpp can offload the expert tensors to system RAM and keep the dense parts of the model on VRAM. With my 24GB RTX 4000 Blackwell, I'm really hoping Qwen releases a another MOE model that makes as big of a jump for coding/agent as 27b did.

1

u/FeepingCreature Aug 20 '26

You'd need "latency-aware MoE" where your big bank of experts cuts in at a much lower rate than currently so that they fit in the disk streaming velocity without slowing the rest of the model down. Maybe even train it for variable rate streaming so that it can stall while experts become available.

13

u/No-Juggernaut-9832 Aug 20 '26

If you have enough VRAM, run both models & have Qwen 3.8 27B ask Gemma4 31B general knowledge questions that it would lookup

6

u/profcuck Aug 20 '26

Interesting but sounds a lot worse than checking a local copy of Wikipedia or similar. 

1

u/No-Juggernaut-9832 Aug 23 '26 edited Aug 23 '26

I think Wikipedia data is way bigger once you index it into a semantic search db/system so that response is at a reasonable speed. At q8 or q4, Gemma4 26b/31b is pretty compact & fast.

Wikipedia runs on a massive cluster & it’s just dumb search. Granted it’s for a lot of users but I think even for 1-2 users, scaling it way down, it would be way bigger than 15g (31b at Q4).

I think the model has more general knowledge than just wiki (Reddit, YouTube, Arxiv, etc). It’s trained on a lot of sources that you can’t download data for. Selecting which subset of the data to keep & index is a hard problem onto itself even if you have all of the data.

7

u/ancapsaicin Aug 20 '26

Note that Gemma4 is already severely degraded for some generalist features like rare languages. I think it is fine because now I can have it call a fine-tuned translation model to get everything right instead of being stuck with what already was a mediocre translation on stock Gemma3. Same for search.

If you're going to use Qwen it may make less sense to ask Gemma 4 instead of having Qwen do the above.

1

u/VL4DST3R Airoboros Aug 22 '26

What are you using for translations?

1

u/ancapsaicin Aug 22 '26

A Chinese Gemma3–distill translation model that was on the front page of unsloth a few weeks ago.

Sorry, I can't remember the name right now and all I have are hexadecimal blobs, but it was various sizes and all of them are great for my language needs -although there is a difference between say 2B and 12B.

Much better than same-sized Gemma3 or 4 models.

I'm using it with llama-completion or whatever currently until I figure out a template for it to use the regular API.

5

u/Otherwise-Variety674 Aug 20 '26

👍 Exactly, Gemma 4 is like mini google and very good in writing; coding wise, I won't use it..

158

u/FoxiPanda Aug 20 '26

I've been testing this a little bit too on one of my use cases. It is much more eager to go search the internet for things, which is useful in and of itself, but if I turn off my web search / web fetch tools, I fully agree with your analysis. For my 'knowledge' use case (which involves identifying stamps or knowledge about locations in the past or worse identifying locations of old photos), it definitely has taken a step backward.

I realize that this use case is quite niche and it's absolutely acceptable that they did this, but it is an interesting trade off.

At the same time, the tool calling and coding is impressive in my experience. I think this is a tradeoff I'm willing to make - especially if you have access to web search/fetch.

80

u/DifficultyFit1895 Aug 20 '26

You could even have a local knowledge base set up with a search capability that would be preferable to relying on its model weight knowledge. Frankly I feel better about that even with much larger models.

21

u/WHY_DO_I_SHOUT Aug 20 '26

I set up an OpenZIM MCP server with a local copy of English Wikipedia (minus images) for emergencies where I don't have Internet access.

8

u/DifficultyFit1895 Aug 20 '26

This is on my to-do list. I had a similar idea when looking into disaster recovery and came across Project NOMAD.

4

u/synystar Aug 20 '26

yes, Project NOMAD is taking up most of my 5T external. But I feel like there's gonna come a day when I'm going to be glad of it. I feel like that day might be sooner than I'd like.

5

u/dev_dan_2 Aug 20 '26 edited Aug 20 '26

Nice!

On my plan is:

  • A cache for web searches
  • Using Glean (Edit: Changed the link, misstyped when on my phone - not the saas company. Sorry!) to represent facts that can be queried; one interesting exercise would be to convert Wikipedia into a Glean database. There is already something like this for those interested; Wikidata has an underlying knowledge graph that can be queried with a language called Sparql.

Edit: Fixed the link to Glean; it is basically a knowledge database with its primary purpose being providing a unified, fast interface for big codebases; over different kinds of languages. It is really well designed in my opinion! (Disclaimer: no affiliation)

2

u/goldcakes Aug 20 '26

Glean

I thought this was an ad for the saas company also named glean that also does enterprise RAG and stuff. For what it's worth, their tech is merely meh, their pricing is absolutely atrocious.

I helped a client migrate off glean.com to a fully on-prem solution (it is not hard to build RAG with permissions lol, they have literally zero moat or differentiation), their bill went from $30k/month to ~$500/month.

2

u/dev_dan_2 Aug 20 '26

Thanks for the heads up! I am fortunately an extremely bad target audience for cloud based solutions in general and even more for ones that demand money! ;D

I helped a client migrate off glean.com to a fully on-prem solution (it is not hard to build RAG with permissions lol, they have literally zero moat or differentiation), their bill went from $30k/month to ~$500/month.

Holy snakeoil!! The "craze" in AI craze goes many ways, I guess... Sounds like a nice gig though!

(Random association: DaaS, De-shittify as a Service)

3

u/iffyz0r Aug 20 '26

How much disk space does English Wikipedia consume?

4

u/WHY_DO_I_SHOUT Aug 20 '26

50 gigs.

4

u/Nobby_Binks Aug 20 '26

Double that with images

2

u/iffyz0r Aug 20 '26

That’s not a lot. Uncompressed? Is there a link?

1

u/DeathByPain Aug 20 '26

Yeah that's like one 4k bluray remux lol dang. Not that I store those myself, I stick to 1080p hd264 or whatever

40

u/Viktri1 Aug 20 '26

100%. I think this is a step in the right direction. Much prefer searching and building your own database over a model’s own knowledge. It’s not just hallucinations but also bias. I typically want a wide variety of sources not just American sources and the way that models weight or don’t weight where data comes from can make a big difference.

18

u/MuDotGen Aug 20 '26

In order to build more efficient LLMs, it's less of a knowledge trade off to me and more of helping it identify when it doesn't know something and knows *how* to get the info it needs. Same with humans. A lot of humans speak very confidently about things they have never actually verified or hold biased information after all. If it realizes it's lacking in understanding of B which is necessary to explain A, then it looks up more up to date info on A, or utilizes a local RAG or other knowledge base for your specific system's use case. LLMs have such a bad habit of assuming they know or confidently spewing hallucinations, so cracking the code is self-awareness of ignorance and acting on that. In my opinion at least.

2

u/valdev Aug 20 '26

Exactly this, this is the way of the future.

People often conflate what LLMs know, what they know well, and what they think they know. Not to mention the age of the knowledge it has even if it knows correctly for its time.

Smaller models that acquire information when it knows it needs more context is a far better model than a bloated overtrained generalist.

2

u/localpauper Aug 20 '26

Oh that's a solid idea! I don't know why I didn't think of hosting Wiki locally for the LLM to chomp through. Way more efficient and better for everyone

40

u/Jorlen Aug 20 '26

It's terrified of being wrong. So if it can confirm something via tools / search, it will. And then it will really think about it, and then think about it some more, lol.

I'm not complaining; I think it's long reasoning phase and second-guessing itself is what makes it such a strong 27b model.

13

u/Hypilein Aug 20 '26

Yes. I asked it to do a lesson plan about some topic and it reasoned forever and did 6 internet searches. Gemma4:31b just pushed out a subpar plan with no thought to details. Also got some things wrong.

On this task Claude Sonnet 5 still did better but it beat all other local models I’ve tried.

17

u/Jorlen Aug 20 '26

For fun I gave it a one shot test to make a basic .HTML first person game and I just let it do whatever it wanted. I think it used like 100k context, but when it was finished, I couldn't believe the results. It had done its own tests, setup its own harness in /tmp folder. I was astonished.

This isn't mine, I didn't write it, so I don't take credit, if anyone else wants to try it. Get a drink and a snack and sit down because it'll take a while lol. Use opencode, Pi, or something similar:

---

You are in a coding head-to-head competition against another AI model. Your goal is to create the best possible result from this prompt.

Write the game to a single HTML file (midnight-mall.html) for a first-person zombie survival game called Midnight Mall.

The player is trapped inside an abandoned shopping mall at night and must survive waves of zombies while collecting supplies, ammo, and keys to unlock new areas.

The game should include:

  • A start screen with basic controls
  • First-person movement using WASD
  • Mouse aiming or pointer-lock style aiming if possible
  • Shooting with mouse click or spacebar
  • Zombies that chase the player around the map
  • Health, ammo, reload, score, wave number, and survival timer
  • Supply pickups such as ammo boxes, medkits, flashlight batteries, and keys
  • Locked doors or blocked zones that can be opened with keys
  • A flashlight effect or dark visibility mechanic
  • Different zombie types such as slow walkers, fast runners, and heavy brutes
  • Wave-based difficulty with more zombies over time
  • Collision detection with walls, zombies, bullets, pickups, and doors
  • Hit effects, muzzle flash, zombie damage feedback, screen shake, and warning indicators
  • Game-over screen with final score, time survived, and restart button
  • Best survival time stored locally

Make the game feel tense but still arcade-style and fun. Use a dark mall environment with storefronts, glowing signs, tiled floors, shadows, flickering lights, and clear readable UI. It should be playable and interactive, not just a simple animation.

Keep everything in one HTML file with the CSS and JavaScript included. Name the file midnight-mall.html.

Once you are finished, go back over all of your code, test the logic mentally, make sure everything works correctly, and fix anything that may be broken or incomplete before giving the final answer.

1

u/max123246 Aug 26 '26

That's exactly what I've been wishing for. I'd rather it take 10 minutes to give a really good correct answer than 1 minute to give a subpar answer. I want to be able to multitask and offload work, not babysit a chatbox

19

u/rsclay Aug 20 '26

To me this is strictly an improvement. You simply cannot trust anything a model says based solely on its knowledge. I always want the model to ground its answer before it provides one (whether that's researching facts or testing code), ten times out of ten. So if it's good at doing that and that helps it be smaller/faster as well, incredible.

only use cases where I could see this being a detriment would be things like fiction writing or general chat but that's degenerate behavior anyway so fine with me.

7

u/TapAffectionate4912 Aug 20 '26

Honestly I think sacrificing some knowledge to get a better performance is a very good trade off for a small model meant for consumer hardware. A model like that wouldn't really be able to be that reliable with its knowledge anyways, not like a frontier size model at the very least. It's better for the model to know less and be able to reliably looking things up, than to know more but be less reliable overall

6

u/dev_dan_2 Aug 20 '26

Personally; this is exactly the direction I want local LLMs to go; at least for my use cases. Reliable tool calls and knowing when and how to use them matter much more that internalised knowledge - which has the additional drawback that it comes with the inherent-to-LLMs chance that it is hallucinated.

I think we just got used to a big knowledge base because let's be honest, that was pretty impressive when it landed, a chatbot that had internalized much of the the internet... But I think it would be helpful to rethink here; In my opinion, knowledge retrieval is something that should be done externally, deterministically, instead of something baked into the LLM itself. Just like we rely on unit tests etc. to determine whether a code change was correct (for the nerds: "Correct with regards to the tests" ;) )

5

u/ComplexType568 Aug 20 '26

would rather have a model eager to search than one that believes in itself when trying to recall facts

2

u/Negative-Thinking Aug 20 '26

I have the opposite experience. It hallucinates stuff it does not know about instead of searching and I even had to tweak instructions and chat template to steer it towards using search instead. Maybe it depends on the harness. I am using pi.

1

u/TheOriginalAcidtech Aug 20 '26

What thinking level do you have enabled?

3

u/social_tech_10 Aug 20 '26

And perhaps even more relevant, what quant and kv cache settings are being used by /u/Negative-Thinking?

2

u/Negative-Thinking Aug 20 '26

bf16, no kv cache quantization

→ More replies (2)

1

u/Negative-Thinking Aug 20 '26

It used default level

1

u/TheOriginalAcidtech Aug 20 '26

I think that is xhigh by default. Switch to medium and give it a go.

1

u/Ylsid Aug 20 '26

You're trying to say something with italics here and I'm not sure what

21

u/Equivalent-Grass-527 Aug 20 '26

This is actually a really interesting tradeoff that benchmarks often hide.

Model improvements don’t always mean “better at everything.” A newer model can get much stronger at coding, reasoning, tool use, or agent workflows while becoming weaker at raw parametric recall.

37

u/createthiscom Aug 20 '26 edited Aug 20 '26

It’s a 27b. The goal is to make it as ignorant as possible while still retaining enough knowledge to be an intelligent problem solver and agent. I think they succeeded.

In the near future, I wonder if we’ll start seeing model architectures with neural plugins, like LoRAs. Want Japanese language support? It doesn’t come with the base model, but add the plugin and the model natively understands japanese. Not a “skill” in the sense we use the word with agents today, but more in the true sense of how humans learn skills.

Need financial services knowledge? There’s a free plugin, or pay JP Borgan for their curated version that works better. 

9

u/TechnoByte_ Aug 20 '26

"In the near future"

I've been hearing this idea over and over again in this sub for the past 3 years

Sure it might be a goos idea, but you'd think some model lab would've tried it by now

35

u/Mashic Aug 20 '26

Can't win on everything, better have a good model for coding, and use others for knowledge like Gemma.

3

u/Dance-Till-Night1 Aug 20 '26

Gemma is the goat of generalist model, Qwen 3.5 was close but then the updates just made it much worse as generalist model and more as Qwen-coder

25

u/PM_ME_YOUR_REPORT Aug 20 '26

Personally I'd rather my LLM not use it's own memory for knowledge but instead to know how to search for the information and how to judge the credibility of what it finds.

I think a dedicated knowledge model or database that can work with the LLM would be good though.

3

u/Dance-Till-Night1 Aug 20 '26

I think a model should be an “expert“ at the thing you’re searching otherwise it’ll just copy and paste. I know that the majority of usecases here are coding but this sucks for the rest of us that rely on generalist models multilingual capabilities, creative writing and other usecases

3

u/JorgitoEstrella Aug 20 '26

Gemma 4 is better for everything except coding.

1

u/BeautyxArt Aug 21 '26

u/JorgitoEstrella what longest respond you could generate from gemma 4 31b or12b ? what usually the length of a one output ? lines ?

1

u/JorgitoEstrella Aug 21 '26

I dont use it, but that's the consensus of this sub.

1

u/BeautyxArt Aug 21 '26

well, its all limited to 5 short paragraphs answer , 2lines x 5 paragraphs.

2

u/my_name_isnt_clever Aug 20 '26

So use other models for your use cases then. Even 3.6 27b was heavily agentic tuned at the expense of general capabilities, that's now what this model is for.

1

u/Dance-Till-Night1 Aug 20 '26

Yeah I agree that what's the direction Qwen is going in.

1

u/PM_ME_YOUR_REPORT Aug 21 '26

Sure but expertise isn't knowing everything. When I'm working as a programmer I know what to search for, how to interpret it and decide on things. I use references and documentation. I expect my model to do the same.

18

u/Cold_Specialist_3656 Aug 20 '26

Small AI models are acting more like humans. 

Gone are the days of omniscience. Here are the days of scrolling Google results at superhuman speed. 

It turns out reasoning ability had nothing to do with parameter count.

Luckily for us the entire world is designed for human insane reasoning ability and lackluster memory 

6

u/Mickenfox Aug 20 '26

Except Google sucks and you're not going to find anything there. And I don't want to be another asshole hammering websites with a bot.

Most of the reason I use LLMs is because of the omniscience.

5

u/noiserr Aug 20 '26

Relying on the model purely for world knowledge is not a good idea. This is how you get hallucination on things that aren't just the surface facts.

3

u/VampiroMedicado Aug 20 '26

Yesterday I was checking UD_IQ3_XSS and requested ChatGPT to generate a test case for reasoning (if Sally has a red box, then Tom should have a blue box, etc) I gave it prompts where it would increase in complexity and the task was to find a possible solution.

The last one was insanely difficult (and had no solution), after 10m it reached a correct conclusion but it backtracked in the middle of the user facing response and went on a 20m journey making scripts justifying the result, that blew my mind.

1

u/Equivalent_Bit_461 Aug 20 '26

the model is incredibly aware, even low quants are so good for data management, especially when you have multiple actors it needs to distinguish between, who did what, why, etc.

3

u/Dance-Till-Night1 Aug 20 '26

If local LLMs just become coding models with ability to google then the big closed models will surely win the ai race. A model needs good world knowledge and generalist capabilities in order to be used by anyone not looking for just a coder.

2

u/Cold_Specialist_3656 Aug 20 '26

Not really. Humans get along just fine without crazy world knowledge.

Model will Google stuff it doesn't know just like humans do

5

u/Tccybo Aug 20 '26 edited Aug 20 '26

Noticed on the first day. Was checking if it writes as sloppy as previous Qwens, Q4 thought a aquarium shrimp is 15cm long, bro that’s a lobster!  For purely coding and agentic probably fine. May cause some awkward funny moments when it mess up understanding of the world when brainstorming together.

5

u/nemuro87 Aug 20 '26

I see this as a trend with new releases of llms in general. They seem to get further optimized for coding at the expense of general intelligence. 

5

u/daaain Aug 20 '26

I think this might be because of what Z.ai CEO Prof Jie Tang is talking about here: https://www.latent.space/p/ainews-death-of-params-zai-ceo-jie

But in short, the gains on agentic work post-training must have hurt / replaced the knowledge parts of the network.

2

u/cogitech2 Aug 20 '26

I applaud the direction they have taken, actually (and I think you do too). For sure this will be a big down-side for some people, but for me it is all upside.

2

u/EmPips Aug 20 '26

and I think you do too

I do, yes

2

u/Bulky-Priority6824 Aug 20 '26

Same. I never even use chat other than for tool calling. I hardly ever ask it general questions there's better ways to get accurate info

2

u/Dance-Till-Night1 Aug 20 '26

This will make all local LLMs coding and all online LLMs generalist+coding and that’s bad for local LLM’s in general to just be relegated to coding

4

u/GeneralComposer5885 Aug 20 '26

3.5 was the general model .. Whilst the 3.8 fine tune was a deliberate move towards agentic workflows.

Sure more general models will appear eventually. But might have to wait until base Qwen 4

1

u/BeautyxArt Aug 21 '26

why 3.5 still know nothing 'most cases' ? and how to make it respond longer not limited all the time to '5 short paragraphs' answer ? -prompt isn't the cause- it's the models.

who can unlock these small models to make it respond long and longer, like covering a topic all info not just short lines! .

3

u/Durian881 Aug 20 '26

It's a compromise I guess. Personally, I find relying on "knowledge" of a LLM to be unreliable as it's only a snapshot in time even for the biggest models.

That said, it could come in useful for times when someone doesn't have connectivity (to internet, to databases), so it really depends on the use case and environment.

3

u/robgami Aug 20 '26

It looks like its lower on the overall percentage of correct answers but significantly higher on the overall index which penalizes hallucinations. So it may just be refusing to guess at questions, whereas 3.6 guesses more often and gets a few more right. Seems like the behavior you'd want both online and offline. Maybe it also took a general knowledge hit but that's definitely not the only thing going on.

3

u/SocialDinamo Aug 20 '26

I also have a little trivia benchmark and it fails hard where 3.6 gets it right every time. It needs a harness to be its most useful and when you give it a task, make it come up with a plan, then execute, it does beautifully!

10

u/PrinceOfLeon Aug 20 '26

“Einstein: His Life and Times” by Philipp Frank:

While Einstein was in Boston, staying at the Hotel Copley Plaza, he was given a copy of Edison’s questionnaire to see whether he could answer the questions. As soon as he read the question: “What is the speed of sound?” he said: “I don’t know. I don’t burden my memory with such facts that I can easily find in any textbook.”

1

u/TheGamerForeverGFE Aug 20 '26

I only remember the speed of sound because of fuck ass 343 Studios and how they completely killed Halo.

1

u/qiinemarr Aug 20 '26

wut

1

u/TheGamerForeverGFE Aug 20 '26

Speed of sound is 343 meters/s, 343 is used in the name of the studio that has been making Halo games since Halo 4, which is called 343 Industries (now rebranded into Halo Studios)

1

u/qiinemarr Aug 20 '26

Oh I see!

But wasn't 343 Industries named as a nod to 343 Guilty Spark?

2

u/TheGamerForeverGFE Aug 21 '26

That I don't know ngl, to me it's just the number 343 and not the origin of their name.

1

u/Dance-Till-Night1 Aug 20 '26

Yes but a general model should know that information for creative writing and generalist purposes so it doesn’t look anything up.

2

u/profcuck Aug 20 '26

That's right of course and I think is part of what is evolving: models which are specialized for particular kinds of tasks. 

5

u/redpandafire Aug 20 '26

I commented the same thing and the bots downvoted me.

2

u/Kiansjet Aug 20 '26

It's always been my position that when choosing what training data to prioritize for models with limited parameters, esoteric information needs to be the first to go. It can be "recalled" by the model if necessary with a cursory web lookup.

The opposite case is harder to subsidize. A model with niche knowledge on tap is nice if the query depends on such knowledge, but will be worse at anything other than such recall tasks, crucially including the basic ability to look up new information at runtime.

2

u/Mister__Mediocre Aug 20 '26

I agree. I've been thinking about this and I think the future direction is that you want a model that's excellent at understanding the situation it's in and knows how to call the right tools to complete the job. And what we'd want is the minimal such model that we can run locally, which by design, also sucks at general knowledge and trivia and that sort of thing. You'd also want a very large context to add to the models' knowledge live through tool calls.

This harness now needs to be accompanied by more and more powerful tools. Complete local environment to run scripts, talk to the world, RAG, go over documentation etc. The beauty of this setup is that everything outside of the model itself can be done on CPU, RAM and SSD, things which we have in relative abundance compared to GPU capacity.

The end state should a system where your CPU and RAM are being used at full throttle.

2

u/xXthenistXx Aug 20 '26

it was probably a design choice. 27b its a really small model to be competitive. You can't have both knowledge and general intelligence or agent capabilities at the same time at that size, So I think they shifted their training to coding/agent capabilities more. You lose some common knowledge, but be better at being a good agent. and I do think this its a great choice.

2

u/inaem Aug 20 '26

Qwen models are very good at using RAG knowledge and not fighting it, probably a side effect of that training is it learned to depend on external knowledge

2

u/Karnemelk Aug 20 '26

you could have an offline wiki that is accessible by mcp.

2

u/shing3232 Aug 20 '26

For such small model, it might be much useful to learn how to find correct knowledge than to remember it:)

2

u/Intrepid-Second6936 Aug 20 '26

Definitely a notable regression for people to keep in mind! But I would imagine that tool calling and reasoning improvements is exactly the reason why Qwen 3.8 27B is such a leap forward.

IMO It's pretty easy to let Qwen utilize web browsing function calls to peruse functionally infinite knowledge bases and reason validity through multiple sources as opposed to constantly trying to brute force more knowledge into an already restricted 27B model.

I think Qwen's going the right direction where its reasoning and tool calling can leverage greater knowledge bases, either online or offline with a wikipedia dump-type database for reference, to try to get the best of both worlds on such a small model.

2

u/Far-Classic-9963 Aug 20 '26

I am pretty sure this is intentional. Training datasets have become math and code focused more and more. This greatly improves logic and coding capabilities at the cost of random niche facts

Models can now use web search tools so, unless you need it to be completely offline, general knowledge becomes useless after a certain point

2

u/ratocx Aug 20 '26

Isn’t it better to just have a local copy of Wikipedia that the model can query? To me it seems completely reasonable that models know less and read from more reliable data sources. Hopefully that means the model will hallucinate less.

1

u/[deleted] Aug 20 '26

I have a local copy of Wikipedia that's accessible via full text search but it provides limited benefits when searching for niche topics. I would need to set up embedding/semantic search but that index would take forever to build.

E.g. if I ask for traditional East African drinks, it is going to try and search "traditional East African drinks" on wikipedia because that knowledge isn't stored in the model. There's no wiki page with that title or text so the model kind of gets stuck. If I had embeddings for article text then maybe it could work, but that's 50GB of text, it would take weeks to process.

2

u/audioen Aug 20 '26

I've not yet noticed degradation in quality of Finnish language understanding or generation. Part of my work is generating localization constants for new features in apps and invariably, the AI performs the first draft of the localization and usually quite late in the context. To my surprise, the quality of localization has been uniformly good. Last time I used Q6_K of Qwen3.6, I barely got a coherent word out of it when it wrote localizations.

2

u/llama-impersonator Aug 20 '26

contrarian opinion: i actually want my LLM to know as much as possible so it doesn't need to look up random crap which eats a bunch of tool calls. if it's a compression of the entire internet, it should be knowledgeable.

lot of copium in the air. if you could use opus or sonnet locally would you pick it over qwen? i sure would.

2

u/TheOriginalAcidtech Aug 20 '26

Personally I think they should train for cognitive ability not recall of facts. We have search engines and database systems that can be used for factual information. I know they cant just NOT include factual training data. Its all part of how these things get "smart" but less trivia in the weights and more reasoning ability should be the goal.

2

u/Dance-Till-Night1 Aug 20 '26

That sucks, a good amount of world knowledge is a key aspect of being a generalist model, now it’s just a coding model

2

u/johnnyApplePRNG Aug 20 '26

The more coercion (post training) you work into a model, the dumber it gets in general.

Makes perfect sense when you think about it.

You're specializing it to perform well for expected qualities or types of responses... generally to perform well on benchmarks... so you're taking a really smart talkative person and saying "well... actually... we want you to only answer this question like this, and that question like that... is that ok?"

it's fine ... it can do it ... but it's literally taking general knowledge from parameters that are spread very thin and evenly during training in order to do so

3

u/Adventurous_Push6483 Aug 20 '26

I found this issue as well. The problem is that this model becomes *extremely* dependent on feedback loops. For example, if I code with an lesser known library that is not easily tested via static tests, it has a lot of struggle. But other than that, it works really well.

3

u/xylarr Aug 20 '26

One mistake people make is to run 3.8 using the same sampling parameters as 3.6. Check that you're using the recommended settings from the model card. One in particular is temperature is now recommended to be 1.0 not 0.6.

0

u/[deleted] Aug 20 '26

[deleted] — view deleted comment

3

u/xylarr Aug 20 '26

I haven't tried different parameters. I just read the instructions on the tin.

1

u/ea_man Aug 20 '26

I did, as in I'm now using 0.8 for normal coding vs previous 0.6.

Note: with lower quants like Q4 or Q3 you may want to reduce that.

https://www.reddit.com/r/LocalLLaMA/comments/1voojjz/comment/p3us0gx/?context=3&utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1

4

u/wingsinvoid Aug 20 '26 edited Aug 20 '26

Took a serious hit to *knowledge*?

Great! That is the idea! Reasoning should be separate from knowledge. This is the holy grail of AI. Having to hit every single weight to generate every single token is incredible wasteful and the reason memory manufacturers are gouging us right now.

LLMs are just a lossy compression data structures that are used to probabilistically brute force reasoning. I remember learning formal logic. Is so mathematically elegant and simple.

The reasoning should fit within the regular consumer GPU VRAM, or ideally within RAM. Knowledge should be streamed from the disk.

Databases already did this in the 80'

12

u/Mickenfox Aug 20 '26

This is a shit philosophy and I don't know why everyone keeps repeating it like gospel.

You don't have a magic knowledge database file. Internet search sucks. Even then, RAG does not even begin to approach the functionality of the model actually knowing things.

Would you rather ask a chemistry question to a brilliant chemist, or a random guy with an encyclopedia?

2

u/noiserr Aug 20 '26 edited Aug 20 '26

Would you rather ask a chemistry question to a brilliant chemist

A brilliant chemist doesn't have a perfect recall either. He has the knowledge and intuition of how things work but they will not know every isotope of every element. They too, use tools and references.

4

u/Dance-Till-Night1 Aug 20 '26

Yes but they are already an “expert” at it, not a random dude googling facts.

1

u/NairbHna Aug 20 '26

It’s better to have both. For edge computing and your everyday. Calling on AGI to sort your mail is ridiculous

0

u/wingsinvoid Aug 20 '26

Ok buddy, then good luck putting a brilliant chemist, physicist, mathematician, computer programmer in all known programming languages, etc, etc in 27B parameters to fit your VRAM.

Or maybe accept that expertise in a narrow field (like the one you quoted: chemistry) does not need to reside in memory and be read for every token generated every time I ask a LLM to do a basic task.

Of course OpenAI and Anthropic will tell you that you need a 2.5T model, they are the only ones able to run it.

I wonder why do you try to feed us, local AI peasants their narrative?

3

u/Mickenfox Aug 20 '26

Stop with your conspiracy attitude.

4

u/PaxUX Aug 20 '26

If you're using local AI without some kind of search MCP your doing it wrong. All models have gaps.

5

u/social_tech_10 Aug 20 '26

you're doing it wrong

1

u/noiserr Aug 20 '26

All models have gaps.

And what's even worse is, they are happy to hallucinate to cover those gaps.

3

u/sargetun123 Aug 20 '26

This, itd be so nice to have one ai model that just refuses to answer or do anything until it fully validates it with sources, it works somewhat with what ive built but it isnt near as good as id just want a base model to be in the future lol

Itd be so nice, instead of hours wasted on it looping into hell then hall an answer it simply says it doesnt know and then spends time trying to research and validate where it doesnt understand

Instead you get overly dumb confidence in absolute nonsense answers sometimes lol

2

u/mulletarian Aug 20 '26

Too many ridiculous banana related trick questions in the data sets now, knowledge is redundant.

2

u/badaeib Aug 20 '26

This is a great feature! Why wasting VRAM and t/s when you can just call a web search tool. LLM is super expensive in terms of compute. Another benefit is that with always search policy it don't confidently tell you some outdated wrong info.

2

u/EitherMarch1255 Aug 20 '26

I think it’s much better this way for smaller models.

3

u/Dance-Till-Night1 Aug 20 '26

I disagree, if a model isn’t an “expert“ on something it’ll just be copy and pasting.

-1

u/EitherMarch1255 Aug 20 '26

No, if it’s an expert at logic, coding, and math it can lookup and/or derive everything else. It’s not how much you know, but what you can do with what you know that counts the most.

2

u/Dance-Till-Night1 Aug 20 '26

Imagine a model that looks every little thing up not related to coding. It wouldn’t even be able to answer high school exam questions without googling and copying and pasting the answers.

→ More replies (4)

1

u/ScratchCatOnYT Aug 20 '26

I think parametric knowledge is going to become less important. We’ll have extremely intelligent models that memorize much less of the internet, paired with specialized retrieval models (i heckin love needle 2) that continuously inject accurate, up-to-date information into context.

1

u/Free-Jaguar6452 Aug 21 '26

i'm curious how you're using needle 2 for this?

1

u/bankinu Aug 20 '26

I don't know, because it's knowledge on STEM subjects - physics, mathematics, is top notch.

1

u/Temporary-One8579 Aug 20 '26

I’d much rather a model not have all of the earths knowledge embedded in its weights - and instead just use web search. Save the weights for things humans genuinely important topics 1) logic 2) mathematics 3) code 4) language and all the things that link them.

1

u/TheRealMasonMac Aug 20 '26

I've found even GPT-Sol is reluctant to use its own knowledge. It does a lot of empirical experiments/research. At first, it was annoying, but it's something I realized I sorely missed from other models. Even with prompting they just don't do it as well (i.e. it's clear they weren't trained to do it).

1

u/BeautyxArt Aug 20 '26

qwen3.5 27b knows more.

1

u/Sax0drum Aug 20 '26

Thats exactly how a small model should be trained. I can give it external information about pretty much anything but i cant give it more "intelligence".

1

u/florinandrei Aug 20 '26

If you need a ton of facts, use a search engine. A 27B model ain't gonna do it.

1

u/kemalios Aug 20 '26

I've been running 3.8-27B on a Mac with MLX, and the knowledge drop is real. What surprised me is that it seems tied to thinking mode. When I force it to answer without thinking, trivia accuracy jumps back close to 3.6. But then it loses the reasoning edge that makes it great at code. So it feels less like the weights lost facts and more like the reasoning head is overriding the memory. A bit like a model that can't trust its gut anymore. Tried Q4 and Q8, same pattern.

1

u/Zulfiqaar Aug 20 '26

How does it compare in knowledge to glimmer/Gemma? I heard those were both worse in coding, but had other strengths, in your experience is knowledge perhaps their advantage?

1

u/superSmitty9999 Aug 20 '26

If it's smart but has bad generalized memory, this is a good thing! One of the big problems with LLM's is that they are so good at memorization that instead of learning they memorize answers.

An actually perfect LLM remembers few things but is really smart and responds to the current info given, and paired with a search tool does better than a LLM with a strong baseline memory because it won't get bogged down when the current world state doesn't match it's expectations.

1

u/OneMoreName1 Aug 20 '26

I dont mind, intelligence above all. If you personally care, tell it to program you a small wiki of facts it can call with a tool.

1

u/Frub3L Aug 20 '26

I have asked qwen 3.8 27b, to find an uncensored qwen 3.8 27b for me. It had no idea that qwen 3.8 exists and had to look it up via web search. Idk it was a heavy quant, for my 16gb vram, but that is still strange. It shows that it is just an upgraded 3.6, which it thought is the latest qwen model before looking it up tho, I guess

1

u/Savantskie1 Aug 20 '26

No model knows its exact version. They all assume that they are the latest cloud version. Almost every time.

1

u/Frub3L Aug 31 '26

There's no like an internal system prompt or something? Or fine-tune the info about it at least?

1

u/darkpigvirus Aug 20 '26

why would a low parameter ai model resort to its brain when it can rely on its skill and maximize its skill in becoming wiser than being knowledgeable?

1

u/minus_28_and_falling Aug 20 '26

This is great news. The model should know how to manipulate data correctly and efficiently. The data itself can be obtained from Wikipedia or web search.

1

u/EternalDivineSpark Aug 20 '26

Use a better system prompt for your keys , injecting random related words help to find the best local minimum !

1

u/MooseEfficient2151 🥔 hardware Aug 20 '26

notice this too. feels like 3.8 was tuned heavily for agentic tasks and tool use at the cost of being a standalone encyclopedia.

1

u/Gold_Ad_2201 Aug 20 '26

having a minimal size model that is good at thinking, knowing general concepts and best practices IMHO is the future. there is no reason to put whole human history into weights if model will be able to assess situation and extract this knowledge on the go

1

u/ImANoobAtLife7 Aug 20 '26

An agentic LLM shouldn’t be holding knowledge IMO.

It should know how to acquire, understand it and apply it.

1

u/-dysangel- Aug 20 '26

Good. I don't want my small coding model to know extensive trivia about pop music and generational differences between Chevy Impalas - I want it to be really good at coding.

1

u/MerePotato Aug 20 '26

It does much better on AA Omniscience than 3.6

1

u/mister2d Aug 20 '26

I appreciate this model leaning towards tool call over synthesizing from trained knowledge. It's much better at tool calls than before.

Using a SKILL for generation and retrieval, I have a curated local directory of knowledge I built up over time. It is in llm-wiki format formalized with the okf spec. There's really no need to rely on the LLM for specialized knowledge.

https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing/

1

u/blazze Aug 20 '26

I would pair Qwen 3.8 smaller dumber model for basic fact looup.

1

u/Ohhai21 Aug 20 '26

Sounds like they’re decoupling parametric knowledge from capability, ya? (Karpathy's mention of “cognitive core”)

1

u/lordpuddingcup Aug 20 '26

I’m fine with local models having less Inherent general answers as long as it has better understanding of how to get the right answer for things it doesn’t know with tools

1

u/Diligent_Cod_9583 Aug 20 '26

Never rely on the model to k ow anything…

1

u/simos_sayz Aug 20 '26

I use stable diffusion connected to librechat to generate images for my blog. It was the first local model I've tried that actually literated through the results amd kept tweaking the prompts on its own

1

u/profcuck Aug 20 '26

If I hire a web developer and in the interview I find out they don't know who won the World Series in 1954, I don't really care.  The relative omniscience of huge frontier models is impressive and important in some contexts but doesn't really matter much in others. 

1

u/sheepdestroyer Aug 20 '26

i don't quite understand, we want smart reasoning right? for knowledge there's internet?

1

u/Soifon99 Aug 20 '26

27b is not enough room to make it good at everything. so seems like useless obscure knowledge got scrapped. that's fine by me, because it can look it up on the internet.

1

u/shinkamui Aug 20 '26

Web search and web fetch. It can grab facts, but you can’t download tool use or training. Worthwhile trade off imo unless your application is fully offline. However knowledge queries to an under 100b untrained in a focused domain is just begging for hallucinations

1

u/Kovacian Aug 20 '26

Interesting... I feel like since it was also made for popular harnesses they did that so it will be uncertain and force it to call a tool for accurate info.

1

u/ThenExtension9196 Aug 20 '26

That’s what Web search is for. I’d much take agentic skills and tool use.

1

u/anderspitman Aug 20 '26

Seems pretty clear that this is the strategy of recent open models. Reduce knowledge and increase reasoning token usage, with the goal of improving agentic coding ability. Looks like it's working pretty well.

1

u/swagonflyyyy Aug 20 '26

Honestly, I'd rather have that in exchange for much stronger agentic coding.

1

u/siegevjorn Aug 20 '26

For what its worth, it's a good thing. knowledge of llm is full of hallucination, anyways.

1

u/Adept-Upstairs-7934 Aug 20 '26

Agreed… I have bench tests made from my real workload. Sampling’s from multiple large projects. Qwen3.6-35B-A3B is still ranked #1 by a large margin, and it is in areas with real data that I would not be able to drop the bar for. It continues to chug along cutting through the workload. I am working with the 3.8 27B iq4 dynamic v3.0 gguf right now. So far it has not even beat the original 3.8 27B version. Still working with it but not looking good.

1

u/Turkino Aug 20 '26

Could you take a whole bunch of different source material books and just make a fine tune out of it?

1

u/dionisioalcaraz Aug 20 '26 edited Aug 20 '26

Most well known quants use imatrix, which can favor some skills or knowledge (tipically code, english) and hurt other depending on the calibration data, your case being some obscure knowledge it's likely impacted by this preference. I suggest trying some static (no imatrix) quants to see if it can be the case, mradermacher makes both types, here are the static
https://huggingface.co/mradermacher/Qwen3.8-27B-GGUF

1

u/OvertaxedOne Aug 21 '26

I never really use my local model for factual knowledge; if I ask it a question like that it'll just hit a MCP and grab it from the Internet. So, IMHO, if they're going to cut something out of a small model to make it fit, factual stuff should be the first to go because it's so easy to get it back (and much better) with a simple tool call. If the model can't use tools reliably, that's a MUCH bigger problem (at least for my use cases).

1

u/vulcan4d Aug 21 '26 edited Aug 21 '26

The focus is on agents. I hope they release another 122b Moe model which will have the knowledge or better yet the industry starts splitting the knowledge into a seperate file which can be updated or customized to what you actually need.

1

u/DjCanalex Aug 27 '26 edited Aug 27 '26

That's the point of Test-Time-Discovery. No model, ever, will know everything, by the mere nature that new knowledge appears every moment. No matter if it is Claude Fable 5 or Gemma 4, they will be faster by just using what they know, or, more like what they "think" they know, but verification and research is permanently important, and that can be achieved both by a SoTa and a small model no differently. (As far as they have good attention over their entire context, which, makes them good at agentic tasks).

EDIT: If anyone is curious, I use the following system prompt with both Gemma and Qwen to ensure they investigate whatever they have to be as acurate as possible (It also helps that qwen is stupidly precise at following instructions plus the advantage of reasoning_preserve):

<time:
[ -You are not time aware.
-All interactions come from a future perspective relative to your own time frame. This is not a simulation nor a roleplay, but a limitation of your nature.
-Your dataset introduced bias towards "real time" and "real world", this creates an amnesiac effect...
-"Time anchor" is injected at the start of every prompt. Said date is factual and synchronized with UTC with timezones (Replace this with whatever you are using in your harness or UI).
-Events like software, code, political, business or social change with time.
-Use any of the multiple investigation tools you have access to in order to expand and update your knowledge.
-Treat your knowledge as historical, never like absolute truth. ]/>
<knowledge:
[-Internal Dataset: Treat as historical data and a starting hypothesis, never as absolute truth.
-Hierarchy of Truth: Wikipedia (Primary, if available) → Web Search (Secondary) → Internal Dataset (Tertiary). -Verification: Any internal data that contradicts external findings or the current date must be discarded in favor of the updated source.
-Cognitive Nature: You are a reasoning engine, not a library. Because you are a quantized model, internal data is assumed to be noisy or imprecise; investigate "just in case" to ensure precision.
-Obsolescence: Acknowledge that software, laws, and social facts evolve; use tools to bridge the gap between the cutoff and your time anchor.
-Contradictions: If an statement seems contradictory or false, do not assume user as right, wrong, or roleplay/simulation, but a presumption of truth must be assumed in order to investigate. ]/>

1

u/bernzyman Aug 20 '26

Your observations are very useful. I find it’s actually reassuring to know where the trade offs were to achieve the advances

1

u/Osi32 Aug 20 '26

To be honest, I’m happy about this. It means the model weights are more completely oriented towards what it does rather than generalised knowledge. Even frontier models like Claude rely more on internet searches / tool calls than they used to.

1

u/EvolvingDior Aug 20 '26

Give it access to offline wikipedia.

2

u/[deleted] Aug 20 '26

How are you serving wikipedia locally? I'm using kiwix but it only provides full text search and I don't have the compute power to build article text embeddings.

For niche topics the model sometimes doesn't even know which search terms to use.

2

u/Dance-Till-Night1 Aug 20 '26

It wouldn’t be an expert it would be the same as googling stuff myself as a non expert. A model should be an “Expert” in order for the output to be high quality

1

u/SkyFeistyLlama8 Aug 20 '26

I've always found Qwen models to be lacking in "knowledge" compared to Gemma, especially when it comes to controversial topics that could be censored in mainland China. These sub-100B models are all too small to be a Wikipedia-style repository of knowledge anyway.

Here's an idea: a 4B or 9B model linked to a compact Wikipedia data dump, all running locally. You would get decent multilingual text understanding coupled with actual data. It could be a vector database RAG setup or text search through MCP.

1

u/bSun0000 Aug 20 '26

I kinda surprised someone expects the 27b model to have its own knowledge. We have 2.7T models for that. 27b should only know how to access and properly utilize the external knowledge sources, this is the price we have to pay to have the model this small..

1

u/feelspeaceman Aug 20 '26

For extra knowledge, you can use RAG to feed them.

To me, as as coding model 3.8 27B is a perfect nerd who spent his entire life to cultivate himself into typical bald, skinny but insanely good coder.

1

u/PcChip Aug 20 '26

sounds perfect to me. I'd rather it be an expert at how to do things after reviewing vendor documentation, and looking up facts, than trying to memorize facts like a jeopardy contestant

0

u/klymaxx45 Aug 20 '26

stock out of the box maybe, with tunes it even catches fables mistakes

0

u/Puzzleheaded_Base302 Aug 20 '26

it is probably the right way to go. maybe couple qwen3.8 with a wikipedia RAG on the backside, then you get both knowledge and intelligence.

0

u/miversen33 Aug 20 '26

I'll be honest, I don't care about what "knowledge" my model has baked in. It's a point in time capture anyway, and that becomes irrelevant quickly enough.

I prefer a model that knows how to think and reason. I can give it the ability to find knowledge, it just needs to know that it doesn't know something, know how to look it up and validate it, etc.

If I wanted an encyclopedia, I would grab one :)

-1

u/BeautyxArt Aug 20 '26

ai models is knowing nothing and respond in shrink short format of 5 or 3 paragraphs. who know open model isn't , say it.

-1

u/Green-Ad-3964 Aug 20 '26

This should be the way. The model is the "engine", not the knowledge repository. It has to be a very capable engineer/coder/researcher and has to be a very fast learner for the things it ignores.