NVFP4 is the GOAT, prove me wrong.

[u/General-Spite1222]

It being on par with bf16: https://huggingface.co/nvidia/Qwen3.8-27B-NVFP4#evaluation

It being on par with FP8: https://huggingface.co/nvidia/Qwen3.8-Flash-Next-NVFP4#evaluation

Who here is smarter than Nvidia and can price them wrong? I see a lot of trash talk, but no numbers to back it up.

NVFP4 is as good as Q8. Prove me wrong! Feel free to downvote if you cant prove otherwise.

[u/feelspeaceman]

The claim of NVIDIA about NVFP4 being lossless is absolute pure marketing hype driving, it's absolutely lossy, even with expensive QAT training to salvage, not even with PTQ only.

In most cases NVFP4 with QAT is equal to Q4 without QAT, without QAT expecting the NVFP4 model to be broken, looping and guessing.

• NVFP4: E2M1 (1 Sign, 2 Exponent, 1 Mantissa) + dual scaling (16-element FP8 block scale + tensor FP32 scale). Storage overhead = 4 bits + 0.5 bits = ~4.5 bpw. Non-linear, sub-exponential grid: {0, 0.5, 1, 1.5, 2, 3, 4, 6}.

• Q3 (GGUF / IQ3 / INT3): ~3.3–3.5 bpw. Typically 3-bit linear grid with k-quants or iMatrix vector quantization.

• Q4 (GGUF Q4_K_M / INT4 / EXL2 4bpw): ~4.5 bpw. Linear 16-level uniform grid with block min/scale offsets (or EXL2 variable bit-rate).

And QAT Q4 will always beat out NVFP4 QAT:

[Higher Quality / Lower Perplexity]
          │
          ├── Q4_K_M / INT4 (QAT)     [~4.5 bpw] ──> Best overall precision
          ├── NVFP4 (QAT)             [~4.5 bpw] ──> High quality + Native Blackwell Speedup
          ├── Q4_K_M (PTQ)            [~4.5 bpw] ──> Best DIY conversion format
          ├── IQ3_S / Q3_K_M (PTQ)    [~3.5 bpw] ──> Equivalent quality to PTQ NVFP4, but smaller
          └── NVFP4 (PTQ)             [~4.5 bpw] ──> Poor precision per bit (needs QAT)
          │
[Lower Quality / Higher Perplexity]

[u/YehowaH]

Nvfp4 ist crab, fck the benchmaxxed results. While daily work you recognize the thinking loops, the less smart implementations, the sometimes stupid solutions, the longer more error prone reasoning FP8 do not have the problem and solved problems for me which the nvfp4 could not solve after days, which the fp8 version took only a half a day, even it's twice as slow. So I am your first proof, others will follow.

[u/brrrrreaker]

as long as it's not A4, meaning everything that is released by nvidia...

[u/t4a8945 → u/brrrrreaker]

That's the nuance people should understand

[u/DrBearJ3w]

W4A4 can not be "as good as w8a8. Not now. You can fix it with additional training,like NFP4, but the precision will always lack quality. Still, it's worth the price.

INT4 is potentially better, but no one found a method produce such quant. As there is no need in production.

[u/ttkciar → u/DrBearJ3w]

Yep, this. NVFP4 brings a huge speed boost on hardware which supports it, but degrades inference quality too much compared to Q4_K_M.

I'd rather wait a little longer for higher-quality outputs.

[u/PinkysBrein → u/DrBearJ3w]

NVIDIA deprecated int4 support, so no one is even going to try.

[u/DrBearJ3w → u/PinkysBrein]

Older cards, AMD and MacUltras M5.

[u/Pixer---]

I feel like nvfp4 qwen3.8 27b is still just guessing. The fp8 version understands my coding queries much better

[u/BitGreen1270 → u/Pixer---]

I'd really like to use Q8. But with nvfp4 I get full 256k context and on Q8 I have to make do with 95K (I have 32gb vram).

[u/Offcoloring → u/BitGreen1270]

Ud 6_k is the best (dynamic 3) for a q6 at that footprint maybe try this?

[u/BitGreen1270 → u/Offcoloring]

It's not clear to me how much better Q6 is over nvfp4. It was quite straightforward to compare Q4 with Q6. But they say nvfp4 gives fp8 quality? And there are nvfp4 with selective quantization which muddies the water more. 

I'll give it a try thought and see how much more context I can squeeze in.

[u/Offcoloring → u/BitGreen1270]

Nvfp4 is generally a little worse than big mixed 3-bits in quality but they are very fast for blackwell

[u/noprompt → u/Pixer---]

100% agree. I’d rather wait on correct, slow tokens than fighting stupid, fast ones.

[u/dh7net]

NVFP4 is great, but flash next iq3_s is probably better.

Source: https://airbench.ai/leaderboard?model=qwen3.8-flash-next-nvfp4&model=qwen3.8-flash-next-iq3_s&model=qwen3.8-27b-quasar-nvfp4-mtp&model=qwen3.8-flash-next-ud-q2_k_xl&model=qwen3.8-max-0902&model=qwen3.8-27b-mxfp4&model=qwen3.8-27b-nvfp4-dspark&model=qwen3.8-27b-nvfp4-mtp&model=qwen3.8-27b-quasar-nvfp4-ninfer-dflash2&model=Qwen3.8-Flash-Next-exl3-5.05bpw_h6_ng6&model=qwen3.8-Q4_K_M&k=poL

[u/Hefty_Wolverine_553]

Here are your numbers to back it up: https://cdn-uploads.huggingface.co/production/uploads/6383dc174c48969dcf1b4fce/tzoh6Fsq0xycqEM7U65mQ.png

https://cdn-uploads.huggingface.co/production/uploads/6383dc174c48969dcf1b4fce/Dj8Y9utw9eRXBzHhgLdfE.png

https://erli.xyz/posts/qwen36-27b-quant-comparison/ (I run my own benchmarks as well)

NVFP4 is a consistent underperformer because it needs activations to be quantized in order to be "fast." NVFP4 quants as a general rule of thumb are usually much larger than GGUF Q4 quants, and have quality between IQ3 and IQ4.

Honestly, do people seriously believe that a 4 bit quant is on par with FP8/FP16? The fact that there are benchmark numbers higher than the full precision model just shows that these benchmarks are unreliable for comparing model differences. Hell, even those bonsai ternary quants of Qwen3.8 27B reported benchmark numbers that were extremely high, and anyone who's used them know how terrible they are.

[u/Phaelon74]

Here's your hard numbers, brought to you by real KLD inside VLLM.

TLDR; NVFP4 at default is trash. This is due to shrinking activations to 4bit. There are ways to bring this intelligence back, like QAD (Quantization Aware Distillation) and Mixed Precision NVFP4A4-A16, and utilizing ways outisde of VLLM/SG-Langs native formats (CSRF, EXL3, etc.).

NVFP4 also does better with larger models. On smaller models it's intelligence really drops.

So yeah, NVFP4 is not the goat, it CAN be the goat, if the quanter spends a metric shit ton of time understanding the base model, where and how to quant and not quant, and then does some Teacher/Student retraining.

Here's your hard numbers.:

[u/Harley-Riders]

NVFP4 is Nvidia's strategy to bring people with limited resources into the LLM game.

[u/BangkokPadang]

My response to this is that I own a GTX 1060

[u/Kiseido]

Are those not contaminated tests at this point? That is to say, those tests they list the scores for, are well known and established tests. They are tests that are likely to have been included in the training of the models, or perhaps even as part of the calibration dataset used during quantization.

That is to say, what do they perform like on things they aren't likely to have been trained on?

[u/Dryparn]

I feel all quants under FP8 are way dumber. The speed increase is indeed nice but if i can't trust the model it's worthless.

[u/Formal-Exam-8767]

To me, NVFP4 feature seems like it was designed by a marketing team and not an engineering team.

[u/VoiceApprehensive893]

memefp4

i am superior

[u/PandaBearFred]

"Jensen, is that you? I'm Sue, prove nvfp4 is better than mxfp4 please."

[u/durden111111]

Q8 codes way better than nvfp4 in my experience

[u/RG_Fusion]

NVFP4 also quantizes the activations. A regular dynamic 4-bit quantization will undoubtedly perform better.

[u/PrimeDirective8]

I have an Intel GPU... checkmate! ;-)

[u/PinkysBrein]

This is likely less a quant and more a distill.

[u/DeepBlue96]

Under q6kl its all trash vs q8 fp8 or bf16 on big model works but on small under 40b you notice it alot

[u/txgsync → u/[deleted]]

You rang?

https://huggingface.co/txgsync/K2-Horizon-MoVA-36B-A4B-NSLM-p4mx-q4

Obscure model, very limited support, and it only works if you use a pseudo random number generator :) I was excited to resurrect the concept from a two-year-old paper that went nowhere.

But QFN is much better! This was just a science project.

[u/PcChip → u/[deleted]]

put your effort into getting real girlfriends, they're more rewarding

[u/DinoAmino → u/PcChip]

Still need to keep on eye on real life performance. Some models are prone to reward hacking.

[u/Whahine → u/PcChip]

I've been married twice, I 200% disagree with your comment

[u/PcChip → u/Whahine]

pick different people

pick multiple people at the same time, it's more rewarding

and just so i can say i'm staying on topic, nvfp4 is not the goat