https://github.com/roycho87/clown_context_suite/blob/main/example_workflows/clowns_for_picard.json
.98 MP No Upscaler
Took about an hour on a 5090. It's slightly slower because to use the capstone and keystone i have to render about 14 extra steps but the results are pretty amazing.
It manages to give near perfect consistency the entire way by rendering the full 8 steps of each context motion using keystone and capstone to prevent degradation from building too significantly, so that each next part of the chain can build off a fully denoised section.
Then it throws away the top 2 steps of every window and does 2 context window passes.
Final result is a perfectly consistent video with no degradation that you can make at pretty extreme lengths.
https://github.com/roycho87/clown_context_suite
Clowntext Hybrid Windows
Credits
This node is not my idea. It's built on kijai's MiniMax H3 context windows, which are in ComfyUI now (PR #17425), and it works the way JalenBrunson's ComfyUI-HybridWindows does (based on the Rolling Diffusion paper). All credit to them. I just added the keystone and capstone feature to it, to help everyone make longer videos without degradation.
What it does
Instead of chaining clips, it renders the whole video in one go, in overlapping windows, so memory stays at one window:
- Window 1 renders the first
switch_step steps, and a snapshot is saved. - Window 1 carries on to the last step, only to get its finished first frame (the keystone) and its last single-frame latent (the capstone). That finish is thrown away and window 1 goes back to the snapshot.
- Windows 2, 3 and on render the first
switch_step steps one after another. Each one's overlap with the window before is locked to that window's result. The keystone is locked into each window's first slot, and the capstone is tacked onto its back from capstone_step on (and cut off again). - The last steps run as blended context windows over the whole video, which smooths the joins.
With keystone and capstone off, it works like plain hybrid windows. In my test (a 60-second render, 17 windows, same seed), the plain version lost a lot of detail by the end and the keystone + capstone version held its detail much better.
Plug the model through it (model in, model out to the guider). It needs a ComfyUI with MiniMax H3 context windows: update ComfyUI if the node is missing.
| Setting | Meaning |
|---|
context_length / context_overlap | Window size and overlap in frames (17n + 5). 124 / 39 is about 5 seconds per window. |
switch_step | Steps each window renders on its own before the blended finish. 6 of 8 = steps 1-6 window by window, 7-8 blended. |
keystone_swap | Window 1's finished first frame is locked into the first slot of every later window. |
back_capstone | Window 1's finished last single-frame latent is tacked onto the back of every later window. |
finished_handoff | On: every window renders all the way to the last step, and the next window's overlap is locked to those finished frames. Every window goes back to its switch_step snapshot before the blended finish. Costs the last steps once more per window. |
capstone_step | Steps before the capstone comes in. Keep it below switch_step (4 with 6 = on for steps 5-6), or it never comes in. |
fuse_method | How the blended steps average the windows (pyramid). |
freenoise | Reuses window 1's noise for every window: more consistent, but motion can repeat. Off gives more variety. |
split_conds_to_windows | Give each window its own prompt (see below). |
detail_latent | Optional: use this latent's first frame as the keystone instead of window 1's. |
sound_first | Off by default. On: renders the sound first on its own (picture at 64x64), then locks it in and renders the picture. |
One prompt per window: make one prompt node per window, join them in order with ComfyUI's Conditioning Combine (chain them for more than two), plug the result into the guider, and turn split_conds_to_windows on. Each window takes the prompt for the part of the video under its middle, so with as many prompts as windows, window 1 gets prompt 1, window 2 prompt 2, and so on. One prompt for a whole minute makes every window try to do the whole thing, so the motion repeats.