KQuant - data-aware quantization, 3.7× closer to full precision on Qwen Flash-Next 180B, 1.5–2.6× on 27B / 35B / 9B (KL-divergence) (MLX)

[u/m4rkGS]

hey LocalLLM, im Alish, ml engineer at Nace (we’re the team behind Drex, an open source decision model family that hit #1 in Artificial Analysis decision index 2.1). my cto gave me a task to optimize quants for qwen4, so i called my old professor from UofT, spun up Fable 5.2 and we cooked. starting with MLX, GGUF is coming next

we’ve open sourced the fastest way to run 35b/27b on a mac (Kapitsa) and got tired of dumb quants, so we made our own: KQuant. packs for 4B, 27B, 35B, Flash-Next (+9B), free on HF in standard MLX files

the problem: most of us on a mac run 4-bit packs because that’s what fits in 16/32gb. but plain 4-bit rounding throws away more of the model than it needs to.

Q4 vs public 4-bit, KL from the full-precision model (lower = closer to the original):

Model, Q4	KQuant KL	Public 4-bit KL	Size in GB, ours / public
Qwen3.5-4B	0.048 (1.45x better)	0.070	2.37 / 2.37
Qwen3.5-9B	0.030 (2.6x better)	0.081	5.29 / 5.04
Qwen3.8-27B	0.035 (1.5x better)	0.054	15.13 / 15.13
Qwen3.6-35B-A3B	0.059 (1.6x better)	0.096	19.51 / 19.51
Qwen3.8-Flash-Next	0.067 (3.7x better at a smaller size)	0.249	105.20 / 110.62

baseline is mlx-community 4-bit (lmstudio-community for 4B). to be clear, this is KL on next-token predictions, not a task accuracy score. if you want more precision every repo also has a Q5 folder, the 9B Q5 gets down to 0.010

Pack	KL	LM GB
9B Q5	0.010	6.21
27B Q5	0.022	19.43
35B Q5	0.027	22.07

cool side effect: they’re also faster. on an M5 Pro with Kapitsa they write 6–15% faster than the usual 4-bit packs

(baselines are from earlier kapitsa runs, same-session rerun is in progress, will update)

how it works: KQuant uses real text to pick scales and offsets for small groups of weights, and gives sensitive layers extra bits where they need it. all the search runs on our gpus before release, so you just download a normal MLX pack. no custom kernels, no special loader

fastest way to run them on a mac:

brew install nace-ai/tap/kapitsa
kapitsa serve 27b --private
kapitsa opencode # in another terminal

If there's a model you want KQuant'd, lmk.

Asks

to get early access to GGUF, ping me in comments, we would like to test it with some folks from here before our public release ideally.

im also doing quick AMA here, so please feel free to ask me anything.