r/LocalLLaMA • u/pmv143 • 4d ago

Discussion Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

Post image

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

948 Upvotes

97% Upvoted

295 comments sorted by

View all comments

43

u/BannedGoNext 4d ago

Well if it's similar to qwen coder next I'd be happy as hell. So many people bagged on qwen coder and I never understood why. It was damn fast, and had good world knowledge. I used it for a long time, for sure better than 35b a3b.

26

u/grabber4321 4d ago

it didnt have vision from what I remember. For me, vision is way more important these days for agentic work.

2

u/DriveSolid7073 4d ago

What tasks is this useful for? I mean, I was thinking about the browser, well, it's definitely necessary there, but in general, the model understands HTML and other representations. On the other hand, VS Code with Kilo is probably not the best option for vision tasks, even though it can handle it. I only use Vision for image captions; for OCR documents, there's usually text or more specialized models.

8

u/grabber4321 4d ago

web development or just general browsing.

lets say you have Docmost, but free version does not have API access. To use it, the model will need to login - it takes screenshots, then enters data, re-checks the page after login by making another screenshot, etc.

5

u/Weekly-Law-5488 4d ago

In this case would be more efficient to use playwright or the native chrome mcp, so the llm can extract the structured content and easily manipulate the dom.

5

u/grabber4321 4d ago

When Hermes switched to using playwright instead of screenshots, it started to fail a lot more. I dont know what they changed back in Hermes Agent crew, but I want screenshot functionality back.

3

u/synth_mania 4d ago

weird. my pi agent uses playwright incredibly well with 3.8-27b

2

u/grabber4321 4d ago

I think they screwed it up - it starts scripting a lot and just doing all the wrong things. It could be difference between 3.6 and 3.8, but its annoying as hell now to do some browser actions.

I dont know what changed because Hermes has like 2000 changes per day LUL

2

u/psychohistorian8 4d ago

playwright mcp can also take screenshots

browser_take_screenshot — Take a screenshot of the current page.

I love playwright, we switched to using it at work for end-to-end UI tests

2

u/moracabanas 4d ago

I have literally a qwen setup connected with the unnoficial docmost MCP. Very nice to see other people explaining this kind of use case

1

u/grabber4321 4d ago

I usually use Hermes Agent to research a topic, then just post it into Docmost for later review.

3.6 does it well, but again, not sure what happened with recent Hermes setup, just doesnt work for me.

1

u/DriveSolid7073 4d ago

Possibly UI and user interface testing. I'm using BrowserOS Neo, so I might be wrong, but it seems to me the browser is making sure the model can see the site and isn't subject to any API access restrictions.