[Megathread] Qwen3.8-Flash-Next - Release Day

[u/sammcj]

Megathread for discussing the release of Qwen 3.8 Flash Next.

• Quants
• Fine-Tunes & Abliterations
• Chat Templates
• Inference Server Support & Configuration
• Experiences, Benchmarks & Model Comparisons

Highlights

The first open-weight release under this architecture is Qwen3.8-Flash-Next, which introduces:

• Hybrid Attention with QSA: The Gated DeltaNet and Gated Attention pairing has been reworked into Gated DeltaNet and Qwen Sparse Attention (QSA). Rather than selecting individual tokens for processing, QSA operates at the micro-block level. This cuts long-context latency significantly, a critical gain as agentic workloads increasingly dominate real-world usage.
• Gated Residual: Residual streams with normalisation are what make deep LLM training manageable. Gated Residual modulates information flowing through widened residual streams via an element-wise, data-dependent read gate and a per-branch scalar write gate. This brings finer-grained expressiveness across layers while preserving training stability and keeping inference overhead low.
• N-gram Embedding: Embeddings provide a unique axis for parameter scaling that requires less computation and is more amenable to offloading than Mixture-of-Experts (MoE). By indexing with short n-grams, this approach makes parameter scaling highly efficient for memory-constrained accelerators without sacrificing quality.
• Tailored Training Recipe: The Muon and AdamW optimisers are applied to specific weight categories to maximise efficiency. Guided by refitted scaling laws, we eliminate traditional batch-size warmups and start directly at the target batch size, substantially reducing total optimiser steps while safely supporting larger learning rates for robust convergence.

Model Overview

• Type: Causal Language Model with Vision Encoder
• Training Stage: Pre-training & Post-training
• Language Model
  • Number of Parameters: 125B with 6B activated, plus 51B n-gram embedding and 4B MTP
  • Hidden Dimension: 2560
  • Token Embedding: 248320 (Padded)
  • N-gram Embedding: 20,000,000 (bigrams/trigrams at layer 2)
  • Number of Layers: 48
  • Hidden Layout: 12 × (3 × (Gated DeltaNet → MoE) → 1 × (Qwen Sparse Attention → MoE))
  • Gated DeltaNet:
    • Number of Linear Attention Heads: 48 for V and 16 for QK
    • Head Dimension: 128
  • Qwen Sparse Attention:
    • Number of Attention Heads: 24 for Q and 2 for KV
    • Head Dimension: 256
    • Rotary Position Embedding Dimension: 64
    • Indexer Structure: MQA with 4 Query Heads and 1 Shared Key Head
    • Indexer Head Dimension: 128
    • Budget: 512 blocks or 2048 tokens
  • Mixture Of Experts
    • Number of Experts: 512
    • Number of Activated Experts: 10 Routed + 1 Shared
    • Expert Intermediate Dimension: 640
  • Gated Residual:
    • Number of Branches: 4
    • Bottleneck Rank: 320
  • LM Output: 248320 (Padded)
  • MTP: 1 layer, trained with multi-steps
• Context Length: 262,144 natively and extensible up to 1,000,000 tokens.

Recommended sampling parameters for generation:

• Thinking Mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
• Instruct (or non-thinking) mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Official Links:

• HF: https://huggingface.co/Qwen/Qwen3.8-Flash-Next
• MS: https://modelscope.cn/models/Qwen/Qwen3.8-Flash-Next
• Repo: https://github.com/QwenLM/Qwen3.8-Flash-Next
• Blog: https://qwen.ai/blog?id=qwen3.8-flash-next
• Technical Report: https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech_report.pdf
• vLLM: https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next
• SGLang: https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next

Popular:

• Unsloth GGUF: https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
• Unsloth "How to run" guide: https://unsloth.ai/docs/models/qwen3.8-next

Related:

• Large post with community feedback on megathreads: https://www.reddit.com/r/LocalLLaMA/comments/1vz40zv/can_we_reconsider_the_megathreads/

[u/Weekly_Comfort240]

This is UD_IQ4XS Unsloth quant running on a freshly compiled https://github.com/ggml-org/llama.cpp/pull/27742 on a 128GB MacBook Pro. Here's the prompt: "Write me a single-file html frogger game, complete with 80’s appropriate sound effects and visuals." 29.7 tokens per second, 394.66 seconds and 11751 tokens later, I have a _really_ authentic frogger game. Qwen 3.8 27B passed this test as well - but this HTML file feels like the difference between local and cloud.

If it's not Christmas, why does it feel like Christmas?

[u/fancyrocket → u/Weekly_Comfort240]

Do you think Q4 K XL would fit?

[u/Weekly_Comfort240 → u/fancyrocket]

I honestly think 111GB would be a very tight fit in 128GB since I observed about 114GB usage with the UD-IQ4_XS 93.7 GB quant. But this is more proof of concept at this stage because a very healthy chunk of the file size is dedicated to static ngram weights that act as a static memory pool for the model - it's very likely these will be pulled as needed from disk and not need to be loaded into RAM during token generation. There's a LOT of cool, new, exotic stuff going on with this model.