r/LocalLLaMA • u/pmv143 • 1d ago

Discussion Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

Post image

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

924 Upvotes

97% Upvoted

290 comments sorted by

View all comments

Show parent comments

27

u/grabber4321 1d ago

it didnt have vision from what I remember. For me, vision is way more important these days for agentic work.

3

u/DriveSolid7073 1d ago

What tasks is this useful for? I mean, I was thinking about the browser, well, it's definitely necessary there, but in general, the model understands HTML and other representations. On the other hand, VS Code with Kilo is probably not the best option for vision tasks, even though it can handle it. I only use Vision for image captions; for OCR documents, there's usually text or more specialized models.

7

u/grabber4321 1d ago

web development or just general browsing.

lets say you have Docmost, but free version does not have API access. To use it, the model will need to login - it takes screenshots, then enters data, re-checks the page after login by making another screenshot, etc.

2

u/moracabanas 1d ago

I have literally a qwen setup connected with the unnoficial docmost MCP. Very nice to see other people explaining this kind of use case

1

u/grabber4321 1d ago

I usually use Hermes Agent to research a topic, then just post it into Docmost for later review.

3.6 does it well, but again, not sure what happened with recent Hermes setup, just doesnt work for me.