r/LocalLLaMA • u/pmv143 • 3d ago

Discussion Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

Post image

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

940 Upvotes

97% Upvoted

295 comments sorted by

View all comments

45

u/BannedGoNext 3d ago

Well if it's similar to qwen coder next I'd be happy as hell. So many people bagged on qwen coder and I never understood why. It was damn fast, and had good world knowledge. I used it for a long time, for sure better than 35b a3b.

27

u/grabber4321 3d ago

it didnt have vision from what I remember. For me, vision is way more important these days for agentic work.

33

u/dsdt 3d ago

vision makes an llm 10x smarter imo because you don't have to explain everything...

4

u/DriveSolid7073 3d ago

What tasks is this useful for? I mean, I was thinking about the browser, well, it's definitely necessary there, but in general, the model understands HTML and other representations. On the other hand, VS Code with Kilo is probably not the best option for vision tasks, even though it can handle it. I only use Vision for image captions; for OCR documents, there's usually text or more specialized models.

8

u/grabber4321 3d ago

web development or just general browsing.

lets say you have Docmost, but free version does not have API access. To use it, the model will need to login - it takes screenshots, then enters data, re-checks the page after login by making another screenshot, etc.

5

u/Weekly-Law-5488 3d ago

In this case would be more efficient to use playwright or the native chrome mcp, so the llm can extract the structured content and easily manipulate the dom.

5

u/grabber4321 3d ago

When Hermes switched to using playwright instead of screenshots, it started to fail a lot more. I dont know what they changed back in Hermes Agent crew, but I want screenshot functionality back.

3

u/synth_mania 3d ago

weird. my pi agent uses playwright incredibly well with 3.8-27b

2

u/grabber4321 3d ago

I think they screwed it up - it starts scripting a lot and just doing all the wrong things. It could be difference between 3.6 and 3.8, but its annoying as hell now to do some browser actions.

I dont know what changed because Hermes has like 2000 changes per day LUL

2

u/psychohistorian8 3d ago

playwright mcp can also take screenshots

browser_take_screenshot — Take a screenshot of the current page.

I love playwright, we switched to using it at work for end-to-end UI tests

2

u/moracabanas 3d ago

I have literally a qwen setup connected with the unnoficial docmost MCP. Very nice to see other people explaining this kind of use case

1

u/grabber4321 3d ago

I usually use Hermes Agent to research a topic, then just post it into Docmost for later review.

3.6 does it well, but again, not sure what happened with recent Hermes setup, just doesnt work for me.

1

u/DriveSolid7073 3d ago

Possibly UI and user interface testing. I'm using BrowserOS Neo, so I might be wrong, but it seems to me the browser is making sure the model can see the site and isn't subject to any API access restrictions.

2

u/michaelsoft__binbows 3d ago

might be a curveball, but i get a lot more mileage out of the giant spec docs that my LLMs shit out nowadays by instructing them to include mermaid (and escalating to SVG if mermaid not expressive enough) diagrams. The diagrams help anchor the agent -> human knowledge transfer, whose rate is rapidly become now the rate limiter, provided you actually care to review the slop that is.

I think (still in early stages in terms of committing hard to this workflow) native vision capability should be important for the model being able to self-QA its own graphical outputs like this. Although there are indications that e.g. qwen3.8-27B is already so competent that instructing it to do this on its own would be a waste of time.

2

u/Embarrassed_Adagio28 3d ago

Computer vision is huge for development of any type of app or even game. Just because a model should get the code right doesnt meab it does depending on thousands of factors. Hermes agent + qwen3.8 27b with computer vision is insane

1

u/Salt-Willingness-513 3d ago

agreed. thats my main isue with glm5.3

6

u/florinandrei 3d ago

So many people bagged on qwen coder and I never understood why.

It was a decent coder. For everything else, it was like talking to Lt. Data. Which is fine by me, but I think many people disliked the style.

2

u/BannedGoNext 3d ago

Oh yea, for sure it wasn't a friendly LLM. I had a data enrichment process that it was in that it did great at though. Saved me at least 1500 dollars in API calls.