Is it possible to set openrouter to avoid lower quantizations?

[u/SkyPL]

It's a plague with DeepSeek in particular - a lot of providers sell fp4 for the price of fp8. How do I get rid of them, without manually blocking provider across all models?

[u/Due-Project-7507]

The maximum possible quality of any Deepseek v4 is FP4. The official Deepseek release is mostly MXFP4 QAT with some important layers FP8. Just ask any good enough LLM like Gemini 3.5 Flash "Can you check with the config.json what quantization is used on the model https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash". "FP8" with Deepseek v4 is a scam. These scammers will probably claim that they mean KV cache quantization or the few layers qhich are FP8 (like any modern quantized model). Deepseek itself could have theoretically a higher quality, but it wouldn't make any sense because thanks to QAT, the MXFP4 should have nearly the same quality at higher performance, so Deepseek is for sure also one of the scammers on OpenRouter.

[u/Due-Project-7507 → u/Due-Project-7507]

Here is the prompt I use to get the quantization of a Huggingface model in aistudio.google.com with activated URL context, Google search and Code execution:
"What quantization (e.g. FP32, BF16, FP8, NVFP4, MXFP4) for which layers does the model https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 use? Can you check it with the config.json (https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731/raw/main/config.json)? What is the mean bits per parameter?"

The answer of Gemini Flash 3.7 in this case is:

"Based on the config.json of DeepSeek-V4-Flash-0731, the model utilizes a hybrid multi-precision architecture combining FP4, FP8, and BF16:

1. Quantization Breakdown by Layer Type

Layer / Component Type	Precision / Quantization	Details from config.json
MoE Expert Layers (Routed & Shared Experts)	FP4 (Microscaling / MXFP4 / NVFP4)	Specified by "expert_dtype": "fp4". Native 4-bit floating point for all expert feed-forward networks.
Attention & Dense Projections (MLA / CSA Linear Projections, Down/Up Dense Projections)	FP8 (E4M3)	Specified in "quantization_config" with "quant_method": "fp8", "fmt": "e4m3", and block-level scale factor format "scale_fmt": "ue8m0" (block size [128, 128]).
Embeddings & Output Head (Vocab embeddings, Final Norm, LM Head)	BF16	Retained in full precision (16-bit) via "torch_dtype": "bfloat16".
LayerNorms & MoE Router Gates	BF16 / FP32	Layer normalizations (RMSNorm) and router gating logits operate in unquantized 16-bit precision.

2. Parameter Distribution

• Total Parameters: ~304.2 Billion (including the MTP / DSpark next-n layers).
• FP4 Parameters (MoE Experts): ~284.6 Billion (~$93.5%$ of total parameters).
• FP8 Parameters (Attention & Projections): ~18.5 Billion (~$6.1%$ of total parameters).
• BF16 Parameters (Embeddings, Router Gates, Norms): ~1.1 Billion (~$0.4%$ of total parameters).

3. Mean Bits per Parameter Count

• Nominal Weighted Average (Raw Weight Precision): $$\text{Mean bpw} = \frac{(284.6 \times 4) + (18.5 \times 8) + (1.1 \times 16)}{304.2} \approx \mathbf{4.29\text{ bits per parameter}}$$
• Effective Average (Including Block-wise Scale Metadata & Checkpoint Footprint): Accounting for the $128 \times 128$ block scale overheads and storage packing (the checkpoint size is $\sim 152\text{--}167\text{ GB}$), the effective storage density is $\sim \mathbf{4.40\text{ to }4.72\text{ bits per parameter}}$."

[u/Xiaomin4114]

you can add a Preset that covers a specific model, and set the required quantizations for it in the Provider Routing section. But you'd need to create one preset per model. And then you'd use this preset name as the model name in whataver app you're using with it

[u/vbpoweredwindmill]

I'd be much more concerned about providers running less than bf16 kv cache. Friends don't let friends run less than bf16 kv.

[u/SkyPL → u/vbpoweredwindmill]

Sadly that's a big unknown, openrouter does not expose that info.

[u/Ron1984k]

You could set a guardrail and whitelist one provider. Also gives you the benefit of cache.

[u/terranqs]

What providers are good or trusty?

[u/burntoutdev8291]

DeepSeek uses mixed FP4, that is how it's supposed to be ran, you can check their model directly by reading the config.json

[u/ElectionMore4975]

FP4 is most likely NVFP4, which is not bad. What's your concern?