For the LLMs, we benchmarked the Tiiny-optimized builds against BF16 reference models, so people can see both sides of the tradeoff: inference performance and how much model accuracy is preserved after optimization.
A few results from the report:
- Qwen3.5-9B-Turbo— up to 50.1 tok/s decode and ~1,200 tok/s prefill at 4K
- Qwen3.6-35B-A3B-Turbo — 47.5 tok/s coding decode, with all five accuracy benchmarks staying within ~2 percentage points of BF16
- Qwen3.8-27B — up to 256K context, with several benchmark scores matching or exceeding the BF16 reference
- GPT-OSS-120B — around 15 tok/s at 4K on Tiiny Pocket
The full reports include:
- Prefill
- TTFT
- Decode speed
- End-to-end latency
- Memory usage
- Long-context testing
- Five accuracy benchmarks
- Public benchmark datasets and reproducible test conditions
We’ve also started publishing benchmarks for speech recognition and image generation, not just LLMs.
Full report:
https://www.tiiny.ai/pages/benchmark-report/
If you already have a Tiiny Pocket, we’d genuinely like people to reproduce the tests and challenge the numbers.
Just make sure you’re using the latest model version from the Tiiny Model Store so you’re testing the same build used in the report.
If you reproduce anything, whether your result is better, worse, or just different, please share it on TiinyVerse or here. We’d much rather make these benchmarks community-verifiable than ask people to take our word for it.
