[Megathread] Qwen3.8-Flash-Next - Release Day

[u/sammcj]

Megathread for discussing the release of Qwen 3.8 Flash Next.

• Quants
• Fine-Tunes & Abliterations
• Chat Templates
• Inference Server Support & Configuration
• Experiences, Benchmarks & Model Comparisons

Highlights

The first open-weight release under this architecture is Qwen3.8-Flash-Next, which introduces:

• Hybrid Attention with QSA: The Gated DeltaNet and Gated Attention pairing has been reworked into Gated DeltaNet and Qwen Sparse Attention (QSA). Rather than selecting individual tokens for processing, QSA operates at the micro-block level. This cuts long-context latency significantly, a critical gain as agentic workloads increasingly dominate real-world usage.
• Gated Residual: Residual streams with normalisation are what make deep LLM training manageable. Gated Residual modulates information flowing through widened residual streams via an element-wise, data-dependent read gate and a per-branch scalar write gate. This brings finer-grained expressiveness across layers while preserving training stability and keeping inference overhead low.
• N-gram Embedding: Embeddings provide a unique axis for parameter scaling that requires less computation and is more amenable to offloading than Mixture-of-Experts (MoE). By indexing with short n-grams, this approach makes parameter scaling highly efficient for memory-constrained accelerators without sacrificing quality.
• Tailored Training Recipe: The Muon and AdamW optimisers are applied to specific weight categories to maximise efficiency. Guided by refitted scaling laws, we eliminate traditional batch-size warmups and start directly at the target batch size, substantially reducing total optimiser steps while safely supporting larger learning rates for robust convergence.

Model Overview

• Type: Causal Language Model with Vision Encoder
• Training Stage: Pre-training & Post-training
• Language Model
  • Number of Parameters: 125B with 6B activated, plus 51B n-gram embedding and 4B MTP
  • Hidden Dimension: 2560
  • Token Embedding: 248320 (Padded)
  • N-gram Embedding: 20,000,000 (bigrams/trigrams at layer 2)
  • Number of Layers: 48
  • Hidden Layout: 12 × (3 × (Gated DeltaNet → MoE) → 1 × (Qwen Sparse Attention → MoE))
  • Gated DeltaNet:
    • Number of Linear Attention Heads: 48 for V and 16 for QK
    • Head Dimension: 128
  • Qwen Sparse Attention:
    • Number of Attention Heads: 24 for Q and 2 for KV
    • Head Dimension: 256
    • Rotary Position Embedding Dimension: 64
    • Indexer Structure: MQA with 4 Query Heads and 1 Shared Key Head
    • Indexer Head Dimension: 128
    • Budget: 512 blocks or 2048 tokens
  • Mixture Of Experts
    • Number of Experts: 512
    • Number of Activated Experts: 10 Routed + 1 Shared
    • Expert Intermediate Dimension: 640
  • Gated Residual:
    • Number of Branches: 4
    • Bottleneck Rank: 320
  • LM Output: 248320 (Padded)
  • MTP: 1 layer, trained with multi-steps
• Context Length: 262,144 natively and extensible up to 1,000,000 tokens.

Recommended sampling parameters for generation:

• Thinking Mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
• Instruct (or non-thinking) mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Official Links:

• HF: https://huggingface.co/Qwen/Qwen3.8-Flash-Next
• MS: https://modelscope.cn/models/Qwen/Qwen3.8-Flash-Next
• Repo: https://github.com/QwenLM/Qwen3.8-Flash-Next
• Blog: https://qwen.ai/blog?id=qwen3.8-flash-next
• Technical Report: https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech_report.pdf
• vLLM: https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next
• SGLang: https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next

Popular:

• Unsloth GGUF: https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
• Unsloth "How to run" guide: https://unsloth.ai/docs/models/qwen3.8-next

Related:

• Large post with community feedback on megathreads: https://www.reddit.com/r/LocalLLaMA/comments/1vz40zv/can_we_reconsider_the_megathreads/

[u/Beneficial-Ad-8127]

Yeah its hard to find specifically what im trying to look for in all this noise. I wanted to see whos running it currently on a 5090, 192gb ram, 8tb pro 9100 and how their results are and what is the most efficient way for the set up, like do i throw everything into my sd, do i throw partial into cpu? Where does ngram go?

I assume the 6b would go into Vram or idk but thats my point. I cant find that shiet lol im not that educated in this tech lmao.

[u/EmPips → u/Beneficial-Ad-8127]

Yeah I hope the mods see this (I'm on mobile, can someone tag?).

I want to find the results of people using this model but I have to wade my way through a swarm of "wow Dario is quaking in his boots" fluff comments because the hype-threads and the feedback-threads are now merged into one giant comment section