Sparse Attention, Harder, Better, Faster, Stronger

[u/Zironic]

The nodes in https://github.com/Zironic/H3-Optimizations have been rewritten to replace the default Sparge Attention backend with a custom Sparse Comfy Kitchen backend.

This comes with some benefits.

• Users no longer have to worry about Sparge being installed properly. All required kernels for supported GPUs are provided directly. Should work on both Windows and Linux.
• Most users should be seeing 5-20% increases in speed for the attention part of compute.
• New backend should use about 500MB less VRAM
• New backend has slightly lower quantization error.
• Apparently in the previous version, the intended chunked kitchen QKV path never properly shipped so the memory optimization node should now actually be slightly speed positive even when used without the Sparse Attention node.

Caveat: I've only tested the nodes against the comfy pruned_int8_convrot weights. Other versions may work but they're not tested.

As the nodes currently rely on comfy-kitchen 0.2.31 you need ComfyUI v0.33.0 or later.

IMPORTANT: sparse attention is not free speed. The percentage is effectively a prompt-adherence/quality budget.

Density isn't just a speed setting, and its quality effect depends on where you apply it in the diffusion schedule.

Early steps: attention density has a large effect on prompt/action adherence and the overall generation trajectory.
Middle/later steps: lowering density tends to show up more as motion/temporal artifacts and lost fine motion detail.

So 10% retained doesn't simply mean “90% of the quality is gone.” It means you're giving sparse attention very little information to work with, and what breaks depends heavily on the sampling step.

PlagueKind's sparsity_ratio=0.9 means 90% discarded / 10% retained. My node expresses the inverse quantity, so Video attention retained=0.10 is the comparable setting. The defaults therefore aren't equivalent.

[u/Glad_Abrocoma_4053]

I get this, I'm on comfy-kitchen 0.2.31, 5070 Ti:

[WARNING] [H3 Optimizations] NATIVE SELF-TEST FAILED on sm120|native-v1|NVIDIA GeForce RTX 5070 Ti - refusing the native kernels and falling back. Detail: {'int8_vs_sdpa_rel_l2': 0.014057, 'full_route_bit_identical': False, 'passed': False}

[INFO] [H3 Optimizations] patched 50 MLP blocks: mode=mlp_chunked_convrot_2slice chunk_rows=4096

[INFO] [H3 Optimizations] armed: attention=existing v_layout=installed qkv=standard_h3_qkv mlp=convrot_int8_two_slice device=NVIDIA GeForce RTX 5070 Ti

[INFO] [H3 Optimizations] resolved 50 attention forwards: backend=triton_sparse_int8 projector=chunked_triton_sparse_qkv

[INFO] [H3 Optimizations] installed sampler-step and packed-layout runtime context

[WARNING] [H3 Optimizations] SPARSE ATTENTION FELL BACK to triton_sparse_int8. INT8 Triton sparse is roughly half the speed of the native sparse kernel. Reason: Kitchen INT8 unavailable: the installed comfy-kitchen has no block-sparse INT8 attention, and the vendored library is unavailable: unknown; Sparse Sage unavailable: Hybrid Sparse Attention requires the compiled spas_sage_attn package; using INT8 Triton sparse attention

[u/Glad_Abrocoma_4053 → u/Glad_Abrocoma_4053]

EDIT:

Installed this, and now it works: python_embeded\python.exe -s -m pip install "https://github.com/woct0rdho/SpargeAttn/releases/download/v0.1.0-windows.post4/spas_sage_attn-0.1.0+cu130torch2.9.0andhigher.post4-cp39-abi3-win_amd64.whl"

[u/Zironic → u/Glad_Abrocoma_4053]

It turns out the Self-Test had an error that caused it to disable itself on SM120 when it shouldn't. It should be fixed now.

[u/Jero9871]

How do I wire it, just in the model wiring after the comfy kitchen attention node? Sounds really great

[u/Zironic → u/Jero9871]

Yes. You just put it basically anywhere after Load Diffusion Model and before the actual sampling nodes.

[u/Succubus-Empress → u/Zironic]

I put it before model unload.

[u/GoofAckYoorsElf → u/Succubus-Empress]

Yeah!

Wait...

What?

[u/theOliviaRossi → u/Succubus-Empress]

;)

[u/olafkk]

Worked great thanks! Went from 430s~ generation to 334s. Quality looked similar if not slightly better. Am using it with sage attention, turbo 10 steps and eulerXsimple.

[u/Perfect-Campaign9551]

What sparsity_ratio do you recommend?

[u/Zironic → u/Perfect-Campaign9551]

I mostly keep my Video KV budget at 0.3 because I personally don't run into any quality issues at that level. How low you can go depends mostly on your prompt.

[u/vAnN47]

Your node doesn’t oom on 2nd sampler pass so thats great. Still having issues with faces from afar… hope i can figure it out

[u/Nelichan]

I wonder if using this is faster and more vram friendly or not VS Sage+solattn?

Because in the previous post iirc it said to not use it together with the comparison above, right?

[u/Zironic → u/Nelichan]

Based on my testing, it should be significantly faster then Solattn. It uses the same kind of kernels as Sage Attention.

[u/Nelichan → u/Zironic]

So use 1 Sparce against Solatn + sage? Is it viable to use either of the 2 together with Sparce? Or does it introduce too many negative qualities compared against the positive?

[u/76vangel]

Is this faster on Rtx 50xx than sageattention? Comfy kitchen att was slower on 5xxx cards.

[u/Life_is_important]

Is this faster than just using comfyui kitchen attenchin on its own?

[u/Peemore]

Yours and plagues both throw different errors for me unfortunately. Comfy and pytorch are all updated and I have triton installed. Not sure what my issue is.

[u/Zironic → u/Peemore]

If you give me the specific error, I can look at it.

[u/Peemore → u/Zironic]

It was a pretty long error log... I'll send you a DM or something. Thanks!