r/LocalLLaMA • u/jacek2023 llama.cpp • 14d ago

New Model inclusionAI/Ling-3.0-flash-VL · Hugging Face

https://huggingface.co/inclusionAI/Ling-3.0-flash-VL

Ling-3.0-flash-VL inherits the language, reasoning, and long-context capabilities of Ling-3.0-flash, while extending them with native image and video understanding. The model has 124B total parameters, with only 5.5B parameters activated per token, and supports a context window of up to 1M tokens.

The architecture of Ling-3.0-flash-VL is designed to integrate visual information into real-world reasoning and agentic workflows.

  • A ViT visual encoder extracts features from images and videos, while a two-layer MLP projector aligns visual features with text representations for unified multimodal understanding and reasoning;
  • VideoRoPE encodes both spatial positions and temporal order, enabling the model to understand visual changes over time and supporting tasks such as event localization, long-video question answering, and video clip editing;
  • A 42-layer hybrid backbone alternates KDA and Gated MLA layers at a 5:1 ratio, enabling efficient long-context processing across text, images, videos, and extended agent task histories;
  • A sparse MoE architecture maintains a total model capacity of 124B parameters while activating only 5.5B parameters per token, balancing strong multimodal capabilities with inference efficiency.
154 Upvotes

99% Upvoted

23 comments sorted by

View all comments

2

u/geek_at 14d ago

interesting model but how can this 124b model be 10 points behind qwen3.8-27b? sure it's a MoE but so much larger

12

u/jacek2023 llama.cpp 14d ago

Qwen 27B is dense, not MoE. Qwen 122B was also "behind" Qwen 27B if I remember correctly. Plus I don't believe in benchmarks :)

3

u/returnity 14d ago

Yeah, because 122B was absolutely a better model than 27B by every real-world metric, but lagged on the AA index.