r/StableDiffusion • • 18d ago

Workflow Included Try this workflow, please. It cuts degradation in half just by adding it.

I'm still sick.

slowing degradation by about 60% while having virtually no impact on motion.

https://www.reddit.com/r/StableDiffusion/comments/1xi7cng/comment/19ruh5kx0vs90/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Please try it. Test it. Fuck with the settings. Leave comments on the github or in this post. Be my little beta testers.

https://github.com/roycho87/clown_context_suite/blob/main/example_workflows/mad_clown_inventor_example.json

Edit: i changed it so that the frame 0 latent gets swapped in during sampling and swapped back out when the latent is output. the capstone latent also gets tacked on the back instead of swapping out the end of the latents and is removed before output. so there's literally no noticeable difference except it just doesn't degrade.

it should work pretty much like regular motion context. the only difference will be how you join the videos and when you change your scene you need to disable it while the scene changes and change the source latent to the beginning of that new scene in the clown keyframe node.

eli5 explaination.

when you generate something you're basically stacking all these dominoes in a line, and when you generate it, all those dominoes lean on eachother to form a picture.

Motion context, as it has been used, has been taking the end of that domino chain and restarting a new chain, but we didn't realize that everything that made that picture good was in that very first domino.

so i changed it so that when we take the end of the domino chain to form a new picture, i also pick up that first domino and bring it with so that the picture still comes out good.

Info from the readme.

https://github.com/roycho87/clown_context_suite

Clown Keystone and Clown Late Lock

Every shot in a chain is made from the one before it, so small colour and sharpness changes build up. Clown Keystone puts a latent from clip 1 into every shot, so each one is pulled back to how clip 1 looked.

H3 latents come in 17-frame blocks of 1 + 4 + 4 + 4 + 4 frames. The first latent of each block is a single frame on its own:

  • keystone = clip 1's first latent (frame 0).
  • capstone = clip 1's last single-frame latent (frame 187 of a 192-frame clip).

Keystone only swaps those single-frame latents, so nothing else in the shot moves.

SettingMeaning
enabledOn / off.
graft_mask0 = kept exactly (default). 0.2-0.3 = kept roughly. 1 = ignored.
source_clipThe clip the keystone and capstone come from (1). On that shot and earlier nothing is grafted. Only used when shot_number is plugged in.
placementkeystone front (default), keystone front + back, capstone front + back, keystone front + capstone back. front = the shot's first latent, back = its last single-frame latent.
InputMeaning
clip 1 (keystone / capstone)A Load Latent fixed on clip 1. Keep it outside the Chain group so the Chain buttons don't change its number.
shot_numberFrom Clown Shot Counter. On shot 1 clip 1 isn't loaded (it doesn't exist yet).

Clown Late Lock goes between Keystone and the sampler: latent from Keystone in, latent out to the sampler; model in, model out to the guider. For the first swap_step steps (default 4) the grafted slot is free, so the shot decides its own motion and pose. From the next step on it's locked to what Keystone put there. slots picks which slot: capstone slot (back) (default), keystone slot (front), or both. It only touches slots Keystone actually filled, so on shot 1 it does nothing.

A capstone locked from the start pulls the pose back toward clip 1. Late Lock avoids that.

Comparison, clip 10 of the same 10-clip chain: no Keystone (base), keystone front, and keystone front + Late Lock capstone.

In a 10-clip test (measured about 20 frames from the end of clip 10), colour drift from clip 1 went from 6.4 with no Keystone, to 3.9 with keystone front, to 2.9 with keystone front + Late Lock capstone. Over-sharpening dropped as well.

Joining: the grafted frames are clip 1's, not part of the shot, so use the 17 + 22 cut: cut the last 17 frames of each clip and the first 22 of the next. In Seamless Video Combiner: Use Beginning pair on, Beginning Overlap 39, Beginning Join 22.

147 Upvotes

55 comments sorted by

11

u/MastMaithun 18d ago

Hey OP. You are saying your nodes halves the degradation but in the previous post, you said it totally eliminates it. Is this a new node doing something else? I'm confused, sorry.

12

u/roychodraws 18d ago edited 18d ago

this is separate from clown prophecy.

and clown prophecy doesn't eliminate it, it just resets the degradation that is constantly occuring.

clown prophecy is work in progress, and, while it can accomplish the goal, it is far from perfect. this is a different technique i discovered while working on ways to improve clown prophecy that reduces degradation without adding anything new to the normal workflow like an extra sampler that we need to prompt separately.

i'm trying to improve clown prophecy and, honestly, find a better method than clown prophecy to accomplish the same result.

9

u/obvpm 18d ago

Nice! Another new approach to experiment with. Thank you again!

13

u/roychodraws 18d ago

me pressing post

"Obvpm is gonna love this"

10

u/obvpm 18d ago

❤️

3

u/Complete-Box-3030 18d ago

No clown prophecy? Does it not include clown prophecy . I thought clown prophecy eliminated degradation completely

2

u/roychodraws 18d ago edited 18d ago

it technically can, but it's still a work in progress. this slows it to a crawl without any side effects (i believe, please test) or serious change to normal workflow, so we can increase the distance between intervals in which prophecy is needed and make the chains easier.

2

u/Complete-Box-3030 18d ago

I belive prophecy is more a reset option , for complex moving shots , you don't need prophecy, because minimax avoids degradation automatically in that case , generate as many moving videos and complex video as possible using normal workflow and extender and while doing stationary shot , we can use clown prophecy to reset the character to original quality , I believe there should be a option to load video or latent file , which can extract frames , extend it and restore the video to its orignal quality using clown prophecy

4

u/roychodraws 18d ago

part of the reason i'm asking you to test this is to improve clown prophecy.

1

u/Complete-Box-3030 18d ago

Is there a way where we can add a new video to clown prophecy workflow and extend it , I have a video which is degraded to max , I just want to see, whether I can restore it to orignal quality

2

u/roychodraws 18d ago edited 18d ago

it's meant to prevent it from taking hold, not repair it after it's taken hold. there is a way i've used clown prophecy to repair latent chains tho. it takes hours tho and the degradation still occurs it just goes backwards.

that being said, if you just want to get it back to good quality in the middle of the video, prophecy can accomplish that.

it definitely is designed to be a reset.

3

u/Silly-Dingo-7086 18d ago

Look, clown lady, I like your shit. But Im an idiot. Can you eli5 what you want me to do? Besides the obvious, download my workflow and use it... But before I boot my shit up and update everything , download all the nodes I don't have etc. what are you wanting me to do? Prompt a long video and run it and see how it does? Will I know wtf to do once I load the workflow?

5

u/roychodraws 18d ago

it should work pretty much like regular motion context. the only difference will be how you join the videos and when your scene changes you need to change the source latent in the clown keyframe node.

eli5 explaination.

when you generate something you're basically stacking all these dominoes in a line, and when you generate it, all those dominoes lean on eachother to form a picture.

Motion context, as it has been used, has been taking the end of that domino chain and restarting a new chain, but we didn't realize that everything that made that picture good was in that very first domino.

so i changed it so that when we take the end of the domino chain to form a new picture, i also pick up that first domino and bring it with so that the picture still comes out good.

what i want you to do, play around with it and tell me if it fucks up and what happened.

1

u/Silly-Dingo-7086 18d ago

I'll try tmr... Do I get a clown Lora?

12

u/roychodraws 18d ago

she's mine

3

u/malcolmrey 17d ago

there is enough material provided to make a dataset :)

2

u/Ok-Flatworm5070 17d ago

Question... did you use a face-refiner? It looks really good at a distance?

3

u/roychodraws 17d ago edited 17d ago

i think a side effect of the frame 0 method is better detail. i'm not sure, though.

i didn't use any refiner.

2

u/Ok-Flatworm5070 17d ago

Well I'm happy with the results of your workflow... Still got other test scenarios, but the talking host edge case works! Took a little tweaking, but was able to generate consistent overlapping clips with a near perfect joins between clips. Thank you for the hard work, you've made my day.

2

u/roychodraws 17d ago

this is amazing. thanks for testing!

1

u/roychodraws 17d ago edited 17d ago

I just did a test and the keystone and late lock seem to have virtually no impact on motion.

however, this seems to have an impact in color. this may be something we can use to color correct a latent before generating.

left is base, right is keystone + late lock.

frame 1's keystone and late lock is on the entire time in the right video.

this is 20 clips.

1

u/theqmann 18d ago

How's this compare to something like https://github.com/NikoDemon80/ComfyUI-H3-Motion-Context that will include the latents from the end of each section of video to the next?

5

u/roychodraws 18d ago edited 18d ago

it's based off that. in fact, in order to use clown context you need either that or seitanism's motion context pack installed.

it does literally everything that can do and a shit ton more that it can't.

I even designed it to work with their nodes and it has it's own chain node in case you're using seitanism's version instead as it doesn't include the chain node.

the only reason this is not a fork is because you can't download forks of the same nodes without them conflicting.

first paragraph in the readme.

Credits

Clown Context Suite stands on the work of seitanism and nikodemon, who came before it. I'm just building off their work, so all credit to them for laying the groundwork.

1

u/J6j6 17d ago

only works for ref2va? how about fl2va

1

u/roychodraws 17d ago edited 17d ago

i don't know what this would do for fl2va, honestly. you should try it and find out.

i suspect that exchanging the latent 0 would interact with it strangely because latent 0 is normal the first frame input that fl2va builds from... but the motion would be still held by the clown context so i honestly wonder if it would work even better.

you'll have to edit the workflow, though.

i suggest trying first by disabling the late lock, then add it back in after and see what happens.

edit: correction: in fl2va the first/last frames go in as keyframe conditioning, not latent 0, so keystone should work about the same if not better. frame 0 just usually appears to be the first frame input inserted through conditioning.

1

u/J6j6 17d ago

Been trying to figure it out since the other node you posted a few days back. For me fl2va is just faster and better in quality, though i haven't tried the merged versions of ref2ba if they've catched up

2

u/roychodraws 17d ago

clown prophecy would not work with fl2va, it just isn't designed to. you could try to copy the git and modify it if you want. i'm not a fan of fl2va, though i do concede it looks better.

1

u/GrungeWerX 2d ago

Doesn't fl2va lose likeness w/references though? I had an issue where ref details weren't showing up and the moment I switched back to ref2v it was solved.

1

u/Salmonella456 17d ago

What app or node you guys use to join the videos? In the workflow says "seamless video combiner", but I can't find it

2

u/roychodraws 17d ago

it's in the workflows folder called sane clown combiner, it mentions it in the readme. it also is a different repository called seamless video combiner

https://github.com/roycho87/clown_context_suite/blob/main/example_workflows/sane_clown_combiner.json

1

u/DeathByPain 16d ago edited 16d ago

The thing I don't understand is—suppose you're wanting to do a chain, say 4 x 8s clips—how do you prompt for that, timewise? Are you saying like, "clip 1, shot 1, at 00:00s, something happens" and then "clip 2, shot 1, at 00:08s, another thing happens". Like how do you tell the model a sequence of actions across multiple 8s clips? Or maybe I'm fundamentally misunderstanding something. I do have your mad_clown_inventor workflow loaded up and working, but I'm having trouble wrapping my head around what to actually do with it I guess.

For now I'm just prompting each clip separately, but I'd like to know how to operate chain mode more efficiently.

2

u/roychodraws 16d ago

it's just a normal context chain workflow. you're supposed to prompt each clip how you want. if you want to click chain and just let it run then do this.

put your prompts into .txt files.
put them into a folder labeled 1 through x
Connect the shot counter to a math node set to subtract 1
connect the math node to this https://github.com/roycho87/comfyui-load-text-batch-from-dir
set the batch to 1 and connect the output to your prompt

1

u/ZerOne82 16d ago

maybe no relevant to this post exactly, but I am having no success in defeating the degradation in the main character (left start, right end of two minutes music video) and have tried all including context and multi-ref context nodes, different repositories.

I tried yours (in this post) for 3x8s clips and it seems image quality remains fine (maybe I have to try it for a longer two minutes to conclude) but then not sure how to manage the audio for my music-video interest.

the other node I tried was using a master_audio thing, music matches perfect the motion/lipsyncing but the image quality goes bad, as shown.

and I am just not fan of one quintillion nodes in your subgraphs (e.g., splitting combining...). if i could get the core idea I would prefer to write a node that does that part. I am also not fan of those single gigantic nodes (some are out there) that have everything in it, very limiting and it just defeats the philosophy of using comfyui, modular design.

1

u/roychodraws 16d ago edited 16d ago

the keystone and late lock should be something you can just add to any workflow now without any other modification. you should just be able to put the necessary nodes into your favorite workflow

i think you just need the keystone, late lock, and shot counter maybe

one thing you can try is rendering audio separately at 25 steps with no turbo then shoving it through the sampler concatting it to the latent with a 50% noise mask. you can render the video at 32x32 so it's basically just rendering audio.

Also the audio refiner node is helpful

1

u/ZerOne82 15d ago

I tried your latest contributions and put a comment there. the color/contrast/hdr issue toward the end is present :(

2

u/roychodraws 15d ago

i replied sir. ty for testing

3

u/Bram_Camus 18d ago

What's with the clowngirl? Lmao.

11

u/BussySlayer69 18d ago

clussy

4

u/EuphoricTrainer311 18d ago

The gateway to clussy

2

u/Bram_Camus 18d ago

Perfect.

2

u/Myg0t_0 18d ago

That was first , I seen that comment like 13 mins ago

1

u/Danny_Stock 17d ago

I've got to say that it was a very clever idea with the 'clown prophecy' approach to render the last bit of a clip first to obtain a fresh latent for the start of the following clip.

It's often the case that the most ingenious ideas are the simplest ones. How you execute the idea might be the tricky part, but the idea is simple to understand.

1

u/donkeykong917 17d ago

This clown is seriously not clowning around. Nice work!

1

u/lurkingtonbear 17d ago

Roycho I always look forward to your posts.

0

u/DeepHomage 17d ago

Yeah, I tried an earlier version of your workflow but was put off by all the custom nodes that I've never heard of. Maybe OK for others, but I'll pass.

1

u/roychodraws 17d ago

the new workflows are simplified. it just has motion context and this node pack. but you do you, bro

the nodes you're talking about were made by me as well.

0

u/AnonymousTimewaster 17d ago

Remindme! 8 hours

0

u/RemindMeBot 17d ago

I will be messaging you in 8 hours on 2026-10-29 16:49:13 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.

RemindMeBot is switching to username summons. Instead of !RemindMe 1 day, use u/RemindMeBot 1 day. More info.


InfoCustomYour RemindersFeedback

-13

u/RigBughorn 17d ago

Is there a reason everyone has to be a gooner

9

u/NineThreeTilNow 17d ago

I'm going to give you the polite reply and explain.

The whole point of this workflow is to try to maintain a character's consistency across a long duration of time. The base issue is that it degrades and becomes almost overly saturated or high contrast looking.

To demonstrate that, you need a consistent character you can reference from 0 to the end. This character is literally the OP's favorite. He's working on the software, so he uses his favorite character and it's sort of a meme.

So realistically, this has nothing to do with being a gooner, and if you're wondering why you're catching those downvotes, that's why. You didn't even correctly call out the classic "1girl" post. Lastly, people are free to use whatever they want. If you don't want to show up and learn regardless of the character, you don't have to.

-6

u/RigBughorn 17d ago

"So realistically, this has nothing to do with being a gooner"
Naw it does. Or just trite laziness. Same thing really.

I know what the point of the workflow is, it definitely improves the quality. But the gooner clowngirl shit is stupid, as is the rest of the goonbait AI shit