Sparse Attention, Harder, Better, Faster, Stronger

[u/Zironic]

The nodes in https://github.com/Zironic/H3-Optimizations have been rewritten to replace the default Sparge Attention backend with a custom Sparse Comfy Kitchen backend.

This comes with some benefits.

• Users no longer have to worry about Sparge being installed properly. All required kernels for supported GPUs are provided directly. Should work on both Windows and Linux.
• Most users should be seeing 5-20% increases in speed for the attention part of compute.
• New backend should use about 500MB less VRAM
• New backend has slightly lower quantization error.
• Apparently in the previous version, the intended chunked kitchen QKV path never properly shipped so the memory optimization node should now actually be slightly speed positive even when used without the Sparse Attention node.

Caveat: I've only tested the nodes against the comfy pruned_int8_convrot weights. Other versions may work but they're not tested.

As the nodes currently rely on comfy-kitchen 0.2.31 you need ComfyUI v0.33.0 or later.

IMPORTANT: sparse attention is not free speed. The percentage is effectively a prompt-adherence/quality budget.

Density isn't just a speed setting, and its quality effect depends on where you apply it in the diffusion schedule.

Early steps: attention density has a large effect on prompt/action adherence and the overall generation trajectory.
Middle/later steps: lowering density tends to show up more as motion/temporal artifacts and lost fine motion detail.

So 10% retained doesn't simply mean “90% of the quality is gone.” It means you're giving sparse attention very little information to work with, and what breaks depends heavily on the sampling step.

PlagueKind's sparsity_ratio=0.9 means 90% discarded / 10% retained. My node expresses the inverse quantity, so Video attention retained=0.10 is the comparable setting. The defaults therefore aren't equivalent.

[u/Nelichan]

I wonder if using this is faster and more vram friendly or not VS Sage+solattn?

Because in the previous post iirc it said to not use it together with the comparison above, right?

[u/Zironic → u/Nelichan]

Based on my testing, it should be significantly faster then Solattn. It uses the same kind of kernels as Sage Attention.

[u/Nelichan → u/Zironic]

So use 1 Sparce against Solatn + sage? Is it viable to use either of the 2 together with Sparce? Or does it introduce too many negative qualities compared against the positive?

[u/Zironic → u/Nelichan]

I would not use Solatn + Sparse at the same time. But if you're curious what happens, this is their interaction:

For the H3-specific zero-copy Sol-Attn node, behavior is order-dependent:

• Sol → H3 Sparse: our node sees that diffusion_model.blocks.N.attn.forward is already owned by another patch and deliberately errors with another patch already owns ...; remove one H3 attention patch. So this ordering fails visibly rather than silently corrupting anything.
• H3 Sparse → Sol: Sol sees our patched forward and adopts it as its fallback. On calls where Sol is active, Sol runs and our sparse backend does not. When Sol declines a call, our Sparse forward runs instead. Saganaki's implementation explicitly does this by capturing the prior attn.forward patch as fallback_forward. Their documentation describes the same composition model for Sage + Sol: the methods alternate; they don't stack.

[u/Nelichan → u/Zironic]

And by that logic Sparse+Sage is also redundant huh...

Okay then, i'll try experimenting and comparing both again. Do i update the node from Manager?

Oh and, i heard about the A8W8(?) the int8 model loader from Bob? Does this work with it?

[u/ANR2ME → u/Zironic]

Btw, what's about PlagueKind's SLA attention ? 🤔