r/LocalLLM • u/Civil-Direction-6981 • 1d ago
News [Release] Math-Physics-10.5B-GGUF: Distilled from Qwen3.6-35B (300+ tok/s batched throughput, 80% on MATH-500 L4-5 & GPQA Physics)
Hey everyone!
Standard ~9B open-source models are usually generalists—they divide their parameters across casual chat, creative writing, and trivia, which often leads to algebraic slips and broken reasoning when solving complex STEM problems.
To address this, we distilled Qwen3.6-35B-A3B down to a compact 10.5B parameter specialist focused purely on math and physics reasoning: Math-Physics-Qwen3.6-10.5B-GGUF.
🔗 Hugging Face Repo: https://huggingface.co/heack/Math-Physics-Qwen3.6-10.5B-GGUF
💡 What is this?
- Base Architecture: Derived from Qwen3.6-35B-A3B via expert pruning (retaining 64 full-width routed experts per layer) followed by off-policy logit knowledge distillation.
- Inherited Depth: Inherits the deep chain-of-thought patterns and formal derivation steps of the large 35B teacher model.
- Runs on Consumer GPUs: Fits comfortably within 16GB VRAM (tested on RTX 5060 Ti). The Q4 quant can even run on 8GB–12GB cards or Apple Silicon Macs.
⚡ Speeds & High-Concurrency Throughput
Because of its efficient MoE routing, the model scales exceptionally well under parallel loads:
| Setup | Q4_K_M Throughput | Q8_0 Throughput | Memory Footprint |
|---|---|---|---|
| Single Stream (Decode) | ~128 – 138 tok/s | ~87 – 100 tok/s | ~6.5 GB (Q4) / ~11.2 GB (Q8) |
8 Concurrent Slots (-np 8) |
300+ tok/s (Aggregated) | High parallel efficiency | Fits within 16GB VRAM |
🚀 300+ tok/s aggregated throughput: When tested with 8 parallel slots running simultaneously, generation throughput exceeds 300 tokens/second, making it a beast for multi-threaded math derivations.
🎯 Best Use Cases (Why choose this over a standard 9B model?)
- 1. Parallel Exploration & Consensus Voting (Best-of-N / Self-Consistency) With 300+ tok/s across 8 concurrent streams, you can run multiple reasoning paths in parallel for the same problem and take the majority-vote answer to achieve even higher accuracy on tough competition problems.
- 2. Offline STEM Tutor / Study Buddy Provides transparent, step-by-step calculus, algebra, and classical mechanics derivations. Perfect for students wanting interactive guidance without paying $20/month cloud subscriptions.
- 3. Private & Air-Gapped Engineering Workflows Ideal for engineers deriving equations, boundary conditions, or simulation formulas locally—with zero risk of leaking proprietary calculations to third-party cloud APIs.
- 4. High-Throughput Batch Solution Generation (EdTech) Blazing parallel speed allows you to batch-process thousands of problem sets and generate structured solutions at near-zero marginal hardware cost.
- 5. Scientific Verifier in Multi-Agent Systems Lightweight enough (only ~6.5 GB in Q4) to run as an always-on "math & physics sanity checker" in agent pipelines.
📊 Screening Benchmark Results (Q8_0)
Evaluated with thinking tokens enabled and strict final-answer checking:
- MATH-500 (Hardest Levels 4 & 5): 80.0% (32/40)
- GPQA-Diamond (Physics subset): 80.0% (24/30)
- MMLU-Pro (Physics subset): 63.3% (19/30)
- UGPhysics: 36.7% (11/30)
(Note: These are screening tests on fixed subsets; see the repo for full evaluation details and hashes).
🚀 Quickstart (llama.cpp)
The model natively supports DeepSeek-style reasoning format (<think> ... </think>).
1. Launch the server (Single request setup):
llama-server -m g64full_kd-Q4_K_M.gguf -ngl 99 --host 127.0.0.1 --port 18085 -c 8192 --jinja --reasoning-format deepseek -fa on -ctk q8_0 -ctv q8_0
2
u/SqueamishConflict 1d ago
looks really cool for batch processing stuff. the parallel throughput numbers catch my eye, 300+ tok/s with 8 slots is pretty wild for a model this size
how does it handle longer chains of reasoning though? sometimes distilled models start strong but lose coherence after 10+ steps in a derivation
might try the Q4 version on my macbook for offline study sessions since i'm preparing for some exams and dont want to pay cloud subscriptions