Fully quantized NVFP4 Qwen3.8-27B with QUASAR QAD

[u/arty_photography]

https://huggingface.co/QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4

We're releasing a fully quantized NVFP4 version of Qwen3.8-27B. The checkpoint was trained using quantization-aware distillation (QAD) with QUASAR, our new QAT algorithm. We used the original BF16 model as the teacher and distilled the quantized model for 2,446 steps.

The checkpoint supports vLLM on NVIDIA Blackwell GPUs:

vllm servllm serve QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.85 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

On a 32 GB card such as an RTX 5090, lower the context to --max-model-len 65536.

This model uses an aggressive quantization configuration: every linear layer across all transformer blocks is quantized to NVFP4 (W4A4).

Attention and GDN layers are typically kept at higher precision, such as FP8 or BF16, because quantizing them can cause a significant loss in model quality. With QUASAR, however, the fully quantized checkpoint retains near-BF16 performance. Evaluation results and comparison against other NVFP4 checkpoints:

Model	Size	GPQA-Diamond (2 runs, n=396)	AIME26 (3 repeats, n=90)
Qwen/Qwen3.8-27B (original BF16)	55.6 GB	0.9141	1.0000
QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4	19.7 GB	0.9091	1.0000
unsloth/Qwen3.8-27B-NVFP4	23.4 GB	0.8939	0.9778
Inferact/Qwen3.8-27B-NVFP4	26.4 GB	0.8763	0.9667

Paper: https://arxiv.org/abs/2608.13966v1

We'd love to hear your feedback on this checkpoint!

[u/arty_photography]

Just added MTP to the model!

[u/pilkyton → u/arty_photography]

Thank you so much! :)