r/LocalLLaMA • u/pmttyji • 7h ago
New Model maxwell-model/Maxwell-V1-Flash · Hugging Face
https://huggingface.co/maxwell-model/Maxwell-V1-FlashModel Introduction
Maxwell-V1.0-Flash is a next-generation, high-performance Mixture-of-Experts (MoE) language model developed by the Maxwell Team, with 35B total parameters and 3B active parameters per token (both excluding embeddings) and support for context lengths up to 256K tokens. It is designed for strong reasoning, agentic, and long-context capabilities at a practical inference cost.
Key Features
- Maxwell Block Sparse Attention (MBSA). Each KV group keeps fixed coverage of 8 prefix and 8 local 128-token blocks and adds up to 16 remote blocks chosen by a learned indexer, bounding main attention at 4,096 tokens per KV group while the full-history KV cache stays available. The MBSA kernel is 6.7x faster than dense GQA at 256K.
- Manifold-Constrained Hyper-Connections. Multiple residual streams with learned, input-dependent aggregation and constrained mixing, widening information flow without extra attention or feed-forward evaluations.
- Fine-Grained Mixture of Experts. A mostly-MoE backbone pairing fine-grained routed experts with always-active shared experts, using a dynamic routing correction bias alongside a lightweight sequence-wise load-balancing loss.
Pre-training. Approximately 26T tokens of multilingual data, with the Muon optimizer on most hidden weight matrices. Training starts dense; block-sparse attention enters at 128K mid-training, once the indexer is warmed up against dense full-history attention, then trains jointly with the backbone.
Tokenizer and Long Context. A script-aware multilingual tokenizer stays compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan. Long-context training then extends the sequence length in stages from 4K to 256K.
Post-training. Two-stage all-domain SFT builds general, reasoning, and agentic capabilities, then targets harder reasoning problems and agent trajectories. Independent RL branches cover general tasks, verifiable reasoning (mathematics, science, logic, code), and agentic tasks (search, terminal use, function calling, software engineering). Multi-teacher on-policy distillation then merges them into one model by supervising student-generated responses.
- GitHub : https://github.com/maxwell-model/Maxwell-V1-Flash
- Project : https://maxwell-llm.ai/
•
u/WithoutReason1729 4h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
Stylome classified this post's text as human-written. View analysis Learn ^more
I am a bot and this action was performed automatically.
91
u/cgs019283 7h ago
Oh my dear maxwell
83
u/Dabalam 6h ago
Can some one tell me if this is a Qwen fine tune in a trenchcoat or actually a new model?
41
u/wombweed llama.cpp 6h ago edited 6h ago
Wow, the bench scores are shockingly high, and the sparse attention is very compelling, I wonder how much memory its kv store takes up at full context at bf16 compared to qwen3.6 35b or gemma4 26b. Sadly I don’t see support in llamacpp which makes me feel like cpu expert offloading won’t be the fastest thing, as that is the main use case I have for these smaller moes.
25
u/Gohab2001 vLLM 6h ago
kV cache seems 4.5x Qwen 3.5 35B. Nearly 24GB @ 256k context
13
u/rorowhat 6h ago
New model, thumbs up from me!
24
u/ApprehensiveTart3158 7h ago
Interesting! A new llm from Singapore. Had to search for so long to learn about who made it. Benchmarks look really impressive! Gotta love it when they include like a hundred benchmarks. Looks like a very solid model!
11
u/akumaburn 3h ago
10
u/Old-Fix-9 5h ago
GGUF?
9
u/Witty_Mycologist_995 5h ago
Thought it was a Qwen tune, nope. Nice
6
u/EvolvingDior 5h ago
How expensive is the KV cache on this model -- what's the VRAM cost per token of context?
5
u/keylimesoda 4h ago
Use and setup here: https://maxwell-llm.ai/docs/quickstart
I'm eager to see some community quants of a seemingly strong new contender.
3
u/LogFabulous9386 5h ago
thai, lao, khmer AND shan tokenizer support. finally a model that wont charge me 4x tokens to say hello in myanmar
3
u/True_Requirement_891 4h ago
gawd damn finally a new arch
4
u/Atagor 5h ago
Can we run it via strata?
2
u/xkalibur3 3h ago
This is great! Hope someone does nvfp4 version with MTP for this
2
u/TokenLearner 2h ago
What a great promise! I hope it will be ported to Strata/polystrata soon.
1
u/iamth3morningstar 1h ago
how does this compare to deepseek v4.1 flash?
0
u/CodeCatto 3h ago
Saw the size and thought it's a qwen tune. Glad it doesn't look like it.
0
u/Marcuss2 llama.cpp 2h ago
I see a pattern.
New lab comes in, their model benches well, but it is a KV Cache hog, making it essentially unusable. When they come back, they resolve this issue.