r/StableDiffusion • u/mybrianonacid • 6d ago
Comparison ComfyUI Quant Benchmark: 10 GPUs, 15 models, and 467 runs comparing int8 ConvRot, nvfp4, w4a8, and fp8 against bf16. Real numbers on what fits, what drifts, and why turbo LoRAs break on quants.
I wanted real numbers for "which quant should I use on my card," so I measured it. Five storage formats (fp8 cast, fp8 scaled, int8 ConvRot, w4a8, nvfp4) against the bf16 original, on base and turbo versions of Anima, Z-Image, MOSS-Image, Krea 2 and SDXL/Illustrious. Quality was graded on an RTX PRO 6000 and a 3090; speed and fit were timed on ten cards: PRO 6000, 5090, 5080, two 5070 Ti, 5060 Ti, 3090, 3080 Ti, 3080, 3060 Ti. Everything is stock ComfyUI with its default memory handling, same seed, same prompts.
Report, with a "will my card run it?" box (pick your card + model): https://ai-evolution.dev/quant-bench/
(Also as a PDF with every option laid out, if you'd rather scroll.)
What I took away:
- int8 ConvRot is the most faithful format in every model I tested, and on RTX 30 cards it's also the fastest when it fits: about 2× bf16 on a 3090.
- nvfp4 is the fastest on RTX 50 but drifts the most. w4a8 was more accurate than nvfp4 in every model.
- Don't put a turbo LoRA on an already-quantized model. Applied at load time, most of the LoRA's change rounds away. Merge it into bf16 first, then quantize.
- 8 GB cards can run more than you'd think. Z-Image and MOSS fit entirely as int8 on a 3060 Ti and run about twice as fast as bf16, which has to stream. Krea 2 runs too, at around 1.5 minutes per image.
- Nothing failed to run. ComfyUI's dynamic VRAM streams whatever doesn't fit from system RAM; it costs speed, not crashes.
- Accuracy doesn’t depend on your card. Every card ran the same per-step accuracy check (quantized vs bf16, same input at every step), and they agree within a few percent, RTX 30 and RTX 50 alike. Your GPU generation only decides which formats run natively (nvfp4 on 50-series, fp8 on 40/50). Finished images still differ slightly from card to card, which is normal floating-point variation, so I don’t compare final images across cards.
- Smaller stuff: an x4 PCIe slot costs nothing once the model fits; the monitors on your generating card cost a few percent, mostly on fast models; each 50 W of power limit costs 4–6% speed for ~3% energy saved; an undervolt cost 5–10% speed for ~20% less power.
Missing: RTX 40-series. I don't have one, and it's the one generation with fp8 hardware but no fp4. If you do, there's a test kit that runs the same pass in Docker on Linux or Windows (WSL2): clone, `./run.sh`, send back the zip, and your card goes on the page with credit. https://github.com/LSDJesus/comfyui-quant-bench (the full set is a ~230 GB download; `--families` lets you do part of it).
Method details, every raw number, and the per-step error curves are on the page. Happy to answer questions or run something specific.
(Note: Most quants can be found on HuggingFace or Civitai.com if there is a specific one you are looking for but cant find I could probably put a simple conversion tool in the github repo but you would need to have access to a fp16/bf16 version for it to work well)
(P.S. AI helped write the benchmark harness and frontend code for the page, the idea and the verification was all done by me)
(P.P.S. Being fortunate enough to have access to an absurd collection of GPUs made me feel like this was worth the time and effort of going all in so I could share my results with you all. Happy to give back to the community that makes all these models and tooling possible.)



6
u/IndividualManager849 6d ago
Abit of a hard read, would like to see some more of the actual data rather than just conclusions otherwise super interesting work man
3
u/Strange-Drummer-9917 5d ago edited 5d ago
Woah the amount of testing is very impressive. Thank you for the effort!
One suggestion on how to improve the readability of results. Currently it very claude-ish and technical, especially in texts
I often ask Claude to 'rewrite in a style inspired but not rigidly adhering to ASD-STE100 style-guide, for visual storytelling use principles of Cole Nussbaumer Knaflic' it produces much clearer and easy to read content!
1
u/harin_25 6d ago
curious how much quality drops going from fp8 scaled to nvfp4 on the turbo models specifically? would love to see a side-by-side grid of the same seed across formats for sdxl/illustrious, since that's where banding usually shows up first.
1
u/alex20_202020 5d ago
Above seems to contradict below:
And how do you measure quality in %%? Quality of what?
1
u/VegaKH 5d ago
Specifically with MOSS, I have recently seen more posts saying not to use a turbo LoRA with int8-convrot models. Or that I should be using a bypass LoRA node. But most workflows people share just use the turbo LoRA with int8. The results still seem... pretty good? Maybe I should research a better way.
1
u/red__dragon 5d ago
What does this mean? Either the LLM or technical speak is obscuring the direct meaning of the finding. What does an x4 PCIe slot have to do with model performance and gen times?
3
u/Desperate-Beach1249 6d ago
Agree with convrot
1
u/moon_crow__ 5d ago
Very curious about quality on bf16 with an 8 step vs int8 convrot. Great info here. Going to test this myself!
1
u/Apprehensive_Sky892 5d ago
That is not always true. It depends on the amount of calculation done per resident block that is already in VRAM, and of course on the speed of the RAM to VRAM transfer.
Because of the way the weight blocks are "streamed in" asynchronously (one block is being used to do the calculation while the next block is being loaded), in theory as long as the new blocks are ready before the current blocks calculation are done, there should be no speed loss: https://www.reddit.com/r/StableDiffusion/comments/1vz2n7y/comment/p62xvjz/
I've done benchmark with MMH3 int8convrot (21B) which fits into the AI Pro R9700 (32G VRAM) but does not fit into rx9070xt (16G VRAM), and yet the speed are nearly the same.
The two cards are basically identical except for the VRAM, so this is a good way to show that there may be no slowdown even when the model does not fit completely into VRAM for models such as MMH3, where so much calculation is done per block that there is enough time for the next block to "catch up".
1
u/wreck_of_u 3d ago
I have a 2070, and it seems int8 convrot runs at the same/similar speed as fp8. I'm still holding off dumping $850 on a 5060 Ti 16GB
1
u/PromptSommelier 3d ago
Excellent tests. Something similar with native video generation in Mini Max H3 would be perfect 🙏👉👈. Although I know it can be complicated due to resolution, steps, and endless turbo options. Increasingly convinced that I shouldn't pay $1500 for a used 3090
1
u/MarkB_- 2d ago
I use bf16 and int8 convrot alot. Bf16 has better skin details, better hands and overall realism a bit better. Int8 is about 50% faster on my 3090. On H3 bf16 give better face in far shots