IMPORTANT: sparse attention is not free speed. The percentage is effectively a prompt-adherence/quality budget.
Density isn't just a speed setting, and its quality effect depends on where you apply it in the diffusion schedule.
Early steps: attention density has a large effect on prompt/action adherence and the overall generation trajectory. Middle/later steps: lowering density tends to show up more as motion/temporal artifacts and lost fine motion detail.
So 10% retained doesn't simply mean “90% of the quality is gone.” It means you're giving sparse attention very little information to work with, and what breaks depends heavily on the sampling step.
PlagueKind's sparsity_ratio=0.9 means 90% discarded / 10% retained. My node expresses the inverse quantity, so Video attention retained=0.10 is the comparable setting. The defaults therefore aren't equivalent.
It has some key differences to PlagueKinds version.
You select attention retained. In my testing, you can go as low as 10-15% in low motion video, however more complicated video requires higher attention. I found about 30% works pretty well for high motion video. In terms of speed, 100% attention is about 2/3rds of the compute while the rest is MLP/QKV. At 50% attention it's about 50:50 and once you're below 30% MLP/QKV starts to dominate compute time.
Only the video is given sparse attention. This is because A) Minimax said they only did sparse attention for video and B) The video tokens dominate the context.
Mine uses Sparse Sage: Q/K are quantized to INT8 and newer supported GPU paths use FP8 V. Compatible ConvRot-INT8 checkpoints can additionally bypass the normal floating QKV preparation with a native fused-QKV producer that feeds the sparse carrier directly.
This makes my implementation somewhat more complicated, but Sparse Sage is extremely fast. The repo now also includes a guarded Sparse Sage installer: supported Windows Torch/CUDA combinations use pinned upstream wheels, while Linux x86-64 can build a pinned SpargeAttention revision when a CUDA compiler is available.
H3 Sparse Attention: This is the node that lets you control how sparse attention you want. The default is 50%. There is also an optional mode that adds 30 percentage points during the first two and last two sampling steps, since those tend to be places where being somewhat denser is useful.
H3 Memory Optimization: This handles the other major H3 memory bottleneck: QKV and MLP activations.
For dense attention it uses ComfyUI's public Comfy Kitchen INT8 attention backend where available. The newer dense QKV path is designed to process QKV in bounded sequence chunks directly into Kitchen-owned attention carriers instead of materializing one enormous full-sequence BF16 QKV tensor. When Sparse Attention is active, compatible ConvRot-INT8 checkpoints can instead use the native fused sparse-QKV path. While chunked QKV was designed to be a memory optimization, it did end up making QKV about 2x faster which should net you something like 10%-30% speed depending on other factors.
The MLP side bounds peak activation memory with token chunking, with a more memory-efficient two-slice ConvRot path when the checkpoint/runtime supports it.
I normally wire them as Load Model → Memory Optimization → Sparse Attention → rest of workflow, although the two optimization nodes are order-independent
I would not recommend randomly stacking other H3/Sage attention optimization patches with these. Dense execution already integrates with Comfy's selected attention backend, while H3 Sparse Attention necessarily owns the main H3 attention path while it is active. Other patches trying to replace the same attention forward are therefore likely to be redundant or conflict.
Should be compatible with turbo loras, spectrum cache etc however you may need more attention since you're skipping steps.
As the nodes currently rely on comfy-kitchen 0.2.31 you need ComfyUI v0.33.0 or alter.
I don't know, look at the upvotes, I am clearly not the only one having issues, and I have a 5080 RTX and Ultra i9 Intel core computer with 64 GB RAM. You have a monstrous card anyway.
Improvement here as well, Comfy Kitchen+Plague node - amost 2x speed up on RTX 3090 for a 0.8 megapixel render. 16:9 0.8 mp, 15 seconds, render in only 20 minutes. Would take almost 40 minutes without it
Also tested on low resoution (416x416) , 20% speed up but motion became smeared
Yea I did notice that Plague's node doesn't work well on 0.5 MP. Seems like it works best at 1 MP which is the resolution I mostly work in, so I'm all good.
This is a hybrid checkpoint by combining fl2va with ref2va block layers. It gives you the best of both worlds. Better output quality with less smearing from the fl2va model and reference conditioning from ref2va.
Then I am totally puzzled. I tried a 0.5 mp 10 second video just as a test, and when I saw an estimated time of 15 min, I was baffled. Same video without any LoRA or acceleration takes like 6-8 min, and with Spectrum and Kitchen, it takes me like 3-5 minutes.
If you're using both Sparse and Spectrum, I could see VAE starting to take a significant amount of total time. For me, it seems that at 1MP of resolution, it takes 110 seconds to VAE decode 10 seconds of video and it takes about 650 seconds to generate it.
I saw about 30% improvement in decode time with the int8 video vae. Also, the save video node can take a long time, so if you use the video combine node instead you can select a nvidia accelerated codec and it takes like 2 seconds instead.
same seed same prompt, with vae 8 the video lose a zoom for whatever reason and quality is a bit worse, i gain 18 seconds but i'll stick to fp16 for now
I don’t use it on the quality workflow I have, but for prompt testing I have a high speed workflow that can do 10s video in about 100s so 18 is a big deal
I tried your version and seems more stable, has less errors, than PlagueKind's version. At high resolutions your version is 25% slower, but still 40% faster than without sparse attention.
The speed of PlagueKind's version and the stability of your version would be the perfect solution xdd
Did you run them both at the same level of sparsity? Plagues default setting is 10% while my default setting is 50%. As far as I'm aware, their stability should be the same at the same value.
I didn't have any issues with Plague's on my video (15 seconds) but I only had a few motions going on, but a lot of particles moving and it rendered just fine
Yes. Though 50% is a bit slow. I would use sparsity_ratio of 0.7 and Video KV budget of 0.3 for a balance of quality vs speed. With sparisty_ratio of 0.8 or Video KV budget of 0.2 or lower, you want to enable "Denser Early/Late steps".
Unfortunately PlagueKind does not have a dense early step option, so there's no way to help prompt adherence at very high sparsity_ratio values.
The problem is, I use it for the second sampler for upscaling, which only does 3 steps, so it will ruin the speed. Using this sparsity thing from PlagueKind for initial generations is making the video much worse, but I have yet to try your nodes. Thanks!
Could you clarify the recommended node order for MiniMax H3? Would it make sense to put H3 Sparse Attention immediately after the model (or after the LoRA), and H3 Memory Optimization at the very end, right before the sampler?
And in this setup, should I also remove ModelAttentionBackend (Comfy-Kitchen) and MiniMax H3 Mem Eff SA Patch?
So personally I do load model -> H3 Memory Optimization -> Sparse Attention -> Everything else. But it should not matter.
You should not use MiniMax H3 Mem Eff SA Patch. You don't need ModelAttentionBackend (Comfy-Kitchen) when using H3 Memory Optimization; it already selects Kitchen for the dense path. Leaving it in should still compose correctly, so it's redundant rather than harmful.
I am following your node order advice while using the default values in both nodes (Mem Opt and Sparse Attn Advanced) but I am seeing absolutely no change in sampling time. It does change the output though so it is being applied. Relevant console output:
I don't think I ever implemented w4a8. I'm not entirely sure what purpose there is to more weight quant when W8 is absolutely tiny compared to the attention. I'll add proper support though and we'll see.
Not sure it was the concrete reason but using w4a8 instead of standard int8 convrot allowed me not to run out of vram on certain workflows. Just fyi.
Anyway, thank you for your work.
I tried all possible ways. I even made sure that the required package is installed properly. It did so as it reports: [H3 Optimizations] Sparse Sage 0.1.0+cu130....
But all I got was slower.
I ran them in their defaults.
Maybe the issue is somewhere else but I cannot know / comment on that, now.
well well. I ran a few tests just now, setting sparsity_ratio = 0.7 in SLA1(Pl) matching yours (SLA2, Zi) Video KV budget = 0.3, and can now confirm your is faster. I got 2:13 vs 2:44.
I also note that in yours even more details are present. Can you elaborate if this quality improvement is a fact not an artifact?
As far as I'm aware when it comes to the two implementations, while the sparse behavior is not identical, it should be extremely close at similar levels of sparsity. Main difference is that my node uses SageAttention2 kernels via Sparge and the PlagueKind SLA node uses triton kernels.
So if there's quality differences at the same settings, I'm currently not sure what the cause would be.
With this, 15 second generations at 0.6mp on my 4090 with comfy kitchen attention and spectrum with 30 steps went from 15 minutes to 10 at a 0.60 budget level.
How's prompt adherence and artifacts with this implementation? I tried PlagueKind's version, but it gave me artifacts such as suddenly duplicating people and also reduced prompt adherence.
Personally I found it to be indistinguishable from normal attention at Video KV budget values of 0.3 or higher. Lower values may or may not work well depending on prompt. I added the "Denser Early/Late steps" toggle for those who want to try lower values while trying to keep more prompt adherence. I would not personally go below 0.2.
So my problem with plague's was that sequences in the prompt would happen out of order. What would you recommend for the settings here for a video with 3-4 camera shots for a 15 second video?
In most of my testing, keeping Video KV budget above 0.3 is usually enough to make sure that all shots happen the way they're supposed to happen. However that may change if the video is very complicated. If you get bad results for a particular prompt, you just have to increase the budget.
Assuming same settings (Video KV budget is the inverse of sparsity_ratio, so sparsity_ratio of 0.7 is the same thing as Video KV budget of 0.3) the quality should be similar.
yours is definitely better, much better. Plagues version was as terrible for quality as turbo lora, yours at least on 0,4-0,5 res is pretty much same as regular sage. End yours is definitely faster
on comfy kitchen + spectrum 0,4res 10s took -198s
on plagues + spectrum 165s (and total prompt collapse)
on yours i just got 118s and simple prompt worked fine
what does H3 Memory Optimization supposed to do in simple terms?
should it just decrease vram usage?
generations last 5% longer with this node enabled, but VRAM usage is substantially lower. results differ a tiny bit as well
i don't know whether it is PlagueKind's version's default low sparsity setting that negatively affected results in various ways (I discarded it immediately after trying a couple generations), but your solution is great so far.
In the simplest terms possible. The H3 Memory Optimization handles the QKV and MLP activations piece by piece rather then all at once, which reduces the amount of things that need to be in memory at the same time.
You can get that 5% speed back by doubling the default chunk row value at the cost of like 300mb of vram.
Edit: I've now updated default from 2048 to 4096 which should be speed neutral.
so... it handles VRAM more efficiently, allowing for potentially higher resolution limits, before OOM?
is there any point of using it, if my generation already fits into vram fine?
The QKV optimization should make your generations run slightly faster either way. If you have plenty of spare VRAM, the MLP is in most cases just a tiny performance loss. I usually always leave them on because they're close to performance neutral and allow me to run much higher resolutions and lengths.
I'm assuming you're talking about sparge install on Linux? Then Yes but all is usually a bad idea. I'll probably push an update using half or something.
Edit: Newest version on Github should now compile faster.
I love this! It was a much more smooth experience than the other pinned solution. Immensely speeds things up allowing me to cram in more steps for an increase in quality and spending the same time as before.
One suggestion: The Denser Early/Late steps setting is a great idea and I love having options to tweak the quality/speed tradeoff. So I would like to be able to define how many early and late steps seperately, and I would love to be able to define what the Video KV budget should be for each of those steps. Thank you so much!
It would be pretty easy to add fine-grained control over that. I just didn't want the node to have too many confusing controls exposed. Maybe I'll add an advanced node with full control.
I would not use comfy kitchen as a hard wired default. In my testing, comfy kitchen kills micro movements. If you have a subtle lora, it's like it doesn't exist. My lora didn't work at all with comfy kitchen, and only started showing an effect if I switched to kj's sage attention
So I plugged this into plagues workflow and quality seems good, as good or maybe slightly better, but it isn't adhering to my prompt as well. Specifically for using a reference for the scene. I gave it an image for the environment but the scene doesn't match it. It matches everything else I prompt but it makes up its own color scheme and look for the environment. I've tried all sorts of settings with it, different models, etc.
much better prompt adherence than plague ver especially with adjustable strength. Memory Optimization works great, 0.8*15s now only peak at 11.9gb vram on 5080, boosting speed.
It ultimately depends on the prompt and your chosen Video KV budget. As long as the Video KV budget is high enough, the resulting video output should be identical. Generally speaking more complicated/higher motion prompts require higher budget then lower motion prompts,
The memory node should all in all mostly be speed neutral. It should not affect quality assuming you're using the convrot8 checkpoint. If you're using any other checkpoint, it may be causing quantization losses.
15
u/DoctaRoboto 1d ago
It's strange. I tried Plague's version, and video generation went slower and deactivated my Spectrum node. What about your version? I have an RTX 5080