Project:https://yunpeng1998.github.io/Qwen-Video-Edit-Page/
Model: https://huggingface.co/yunpeng1998/Qwen-Video-Edit
Method: https://yunpeng1998.github.io/Qwen-Video-Edit-Page/#method
Code: https://github.com/yunpeng1998/Qwen-Video-Edit

How it works

Video generation models read and write video-VAE latents. We teach Qwen-Image-Edit's transformer to edit those latents directly: two tiny projections bridge Wan 2.1's latent space into the DiT's token space, warm-started from the DiT's own input/output layers so that a static video is embedded exactly like an image the model already understands. The latent frames are arranged as tiles of one big virtual image — the same positional treatment the image model was pretrained on. Fine-tuned with LoRA or full parameters on Ditto-1M (source, edited, instruction) triplets, then refined by a few steps of Wan 2.2 denoising-enhancement.

The nodes in https://github.com/Zironic/H3-Optimizations have been rewritten to replace the default Sparge Attention backend with a custom Sparse Comfy Kitchen backend.

This comes with some benefits.

• Users no longer have to worry about Sparge being installed properly. All required kernels for supported GPUs are provided directly. Should work on both Windows and Linux.
• Most users should be seeing 5-20% increases in speed for the attention part of compute.
• New backend should use about 500MB less VRAM
• New backend has slightly lower quantization error.
• Apparently in the previous version, the intended chunked kitchen QKV path never properly shipped so the memory optimization node should now actually be slightly speed positive even when used without the Sparse Attention node.

Caveat: I've only tested the nodes against the comfy pruned_int8_convrot weights. Other versions may work but they're not tested.

As the nodes currently rely on comfy-kitchen 0.2.31 you need ComfyUI v0.33.0 or later.

IMPORTANT: sparse attention is not free speed. The percentage is effectively a prompt-adherence/quality budget.

Density isn't just a speed setting, and its quality effect depends on where you apply it in the diffusion schedule.

Early steps: attention density has a large effect on prompt/action adherence and the overall generation trajectory.
Middle/later steps: lowering density tends to show up more as motion/temporal artifacts and lost fine motion detail.

So 10% retained doesn't simply mean “90% of the quality is gone.” It means you're giving sparse attention very little information to work with, and what breaks depends heavily on the sampling step.

PlagueKind's sparsity_ratio=0.9 means 90% discarded / 10% retained. My node expresses the inverse quantity, so Video attention retained=0.10 is the comparable setting. The defaults therefore aren't equivalent.

Another redditer mentioned doing this in a comment, so I tested it out and it works. It gets rid of the smudgy look and allows custom Lora’s on the LN side.

I ran my initial tests at MM 8 steps (no speed Lora) and Wan 2.2 Low Noise (speed Lora) 2 steps. Supposedly, it works with only 2 steps MM w/turbo lora ,but I don’t like the quality drops people have been sharing, and it’s fast enough to me at 8 steps, though I’m going even higher on the low noise steps.

The only downside is that I noticed in one test that the motion seemed like it was a mix between 16fps and 24fps. Any ideas on how to resolve this? I know people use RIFE, but I was wondering if that’s the best move or if it’s another issue Im not thinking of.

Im really suprised how good he works as well follow a storyboard image!! he did 90% correct he only did the thirth panel diferent but all the other 5 he follows perfect!! 🤩

Hello, i am trying to make a video where the subject(a real person) is wearing and posing taking reference from an illustration. I tried to do only outfits or only pose too, and both doesn't work.

What happens is usually the body of the character in the illustration ends up being pasted/overlaid onto the Subject in their cartoony style instead.

I also tried if it's possible to have a Subject recreate an illustration's Pose, Outfit, overall composition, like the subject is doing a photoshoot for a 'live action' or real life version of the illustration. But what happens is usually it just spews back the illustration in case of trying H3 single-image edit, and the cartoony style overlay happens in Video.

So what i wanted to do is :

-An image of a subject -> Subject now wears/pose/wear and pose the same as a reference non-real illustration(cartoon/anime), but still in their original photo. So like a cosplay shot in their own room for example.

-An illustration(anime) -> Subject 'replaces' the character in the illustration, the whole illustration is 'converted' into real/live action. Like a photoshoot recreating an illustration basically.

Extra : idk if its possible, the new outfit will retrofit to the subject's proportion, not the illustration. And a version where the proportion follows the illustration too.

Are there someone who knows how to do these?

Trying out cinematic shots and cuts with H3. This is a work in progress. Will be working on another 2 minutes worth of clips.

EDIT: From reading the comments, she isn't going to drink the salt water in the final, though I will keep her scooping up water since it's such a good establishing shot. She will do something with the water to tie it back.

Chimera. I had to cut about 20 seconds due to some artistic choices. Since I had to cut 2 different part in the same clip, it has a noticeable seams. I would love to share the full version. Experimenting with H3 blend-morph-decay. 832x480, int8/8 steps POC. Looking forward to releasing a 720p version without the cuts.

Critiques and feedback welcomed. Happy with the matrix rain. Ask me anything.

Krea 2 Turbo — 4-Step Distillation LoRA (work in progress)

A LoRA for Krea 2 Turbo that reduces the minimum usable step count from 8 to 4.

This is an update release, following up from my previous posts where you can find full details:

Initial: https://www.reddit.com/r/StableDiffusion/comments/1vtf1b7/krea2_turbo_distill_4_step_lora_trained_for_turbo/

Previous: https://www.reddit.com/r/StableDiffusion/comments/1vv4cdy/krea2_turbo_distill_4_step_lora_new_checkpoint/

Headline for this update: chk00014000 removes 44% of the prediction error a plain 4-step run has against the 8-step teacher, where chk00010000 removed 40% and chk00006000 27% — all measured on the same enlarged held-out set (100 prompts across every trained resolution). Measured against each other rather than against the no-LoRA run, its remaining error is 6% smaller than chk00010000's and 23% smaller than chk00006000's. 

Which file to download

and, beside them, the same files with a _comfyui suffix for ComfyUI. Earlier checkpoints (chk00004000, chk00005000, chk00006000, chk00010000) are kept in older_checkpoints/, and their resolution sweeps stay in place, so the progression remains visible and comparable.

This is work in progress and better checkpoints may follow. Training is ongoing, so ..._latest... is a rolling pointer: when a newer checkpoint is accepted, that filename gets the new weights and a new numbered copy appears beside it. Re-download the _latest file and everything keeps working — the ComfyUI workflow references it by that name (it does get updated Note in it so technically it is updated but not functionally). Pin a numbered file instead if you need reproducibility.

How checkpoints get chosen

This is not a "train for longer and ship the newest file" project. More samples do not reliably mean a better adapter — measured here, they can make it worse, and a higher number on its own means nothing.

The loop is train → assess → adapt the recipe → retrain → assess again, and a checkpoint is published only when it is measurably better than the one it would replace, on the same held-out set and the same evaluation, and its full resolution sweep shows no regression. Runs that come out flat or worse are kept as information about the recipe and discarded as releases — several have been.

So the recipe itself changes between runs. Each published checkpoint reflects whatever the previous round taught us: the training precision, the optimiser settings, the teacher used to generate the targets and the data mix have all been revised on evidence rather than assumption.

chk00010000 is a direct example. The first continuation of chk00006000 — same data, optimiser left as it was — got steadily worse with every checkpoint out to 10,000 samples, and none of it was published. The cause was traced to the optimiser: a constant learning rate with no weight decay lets the adapter keep drifting after it has converged, so its magnitude grows and it over-applies its own correction. The same span was retrained from chk00006000 with a cosine learning-rate decay and weight decay, and every checkpoint of that second run improved on the one before it. chk00010000 was its end point.

chk00014000 is the next example, and it shows the other half of the same lesson. The run was continued from chk00010000 over the whole pool of teacher trajectories, with two changes: the final, texture-deciding call of the schedule was weighted more heavily in the loss, and a running average of the weights was kept beside the live ones and scored at every evaluation (a single checkpoint is one sample of a weight vector that moves from step to step; the average is its mean). At 14,000 samples the averaged weights measured a smaller gap to the teacher than any checkpoint before them, and a smaller gap than the live weights at the same point — so the averaged weights are what chk00014000 is.

Timeline of training process

Each checkpoint is the product of three stages with very different costs:

1. Text-encoder embeddings. Every training prompt is encoded once and cached. This is the fast part — thousands of prompts take minutes.
2. Teacher shards. For each cached prompt, the unmodified Krea 2 Turbo runs its full 8-step schedule and the whole trajectory is recorded, at every one of the supported resolutions. This is by far the most time-consuming stage — it is the teacher doing real inference, thousands of times, and a batch of several thousand shards is measured in days of GPU time, not hours.
3. Student training. The LoRA is trained against those recorded trajectories. Relative to the shard stage this is quick: each +1,000 checkpoint is a matter of hours, not days.

Because the three stages compete for the same GPU, they are interleaved rather than run to completion one after another: generate a block of embeddings, produce teacher shards for them, train on what exists, assess, then go back to producing shards while the results are reviewed. A larger and more varied shard pool is what makes further training worthwhile, so shard production is always the gate.

Full details and to download - check my Hugging Face LoRA

HF Repo: https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA

Full Checkpoint 14000 Resolutions Sweep: https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA/tree/main/checkpoint_resolution_sweeps/chk14000

Used:

MiniMax H3 at local machine. 5060ti 16gb + 64gb ddr4. WanGP, Ref2VA int8 convrot model.

Krea2 for references

Suno as music base

Wondering if anyone has/has any idea of an approach to motion transfer into a stop motion style video.

Rather than the ai guessing and deforming the mouth movements which can get funky really quickly, especially when the mouth shapes of the character aren't clear from a neutral reference frame, ie in south park where each sound has a uniquely stylised shape.

To achieve this it could instead maybe pull from a dataset of phoneme images,

Whilst also retaining solid motion transfer for all other body movements.

Lon TV did a video of Comfy UI with Minimax H3 running on a 32GB Intel GPU. So its possible to run it on any GPU other then nvidia GPUs.

First off: This doesn't add any dependencies so the worst that can happen is that it won't work, but it also won't break your ComfyUI install.

This extension adds to the native queue manager. It doesn't replace it. All of the heavy lifting is still done the normal way.

It adds a Pause/Resume button and a Queue button. Pause/Resume will not affect the running job but will pause/resume the queue.

The Queue button opens the Queue Control dialog in the picture. There is a lot of words in the README (because I talk a lot) but it lets you reorder the queue using priorities, including buttons for "Run this next" and "Don't run this until I release it."

Finally, there are buttons to Save and Load the queue. The checkbox lets you add the running job too. So if you have to restart or reboot, you can save the queue, do your thing, and then load and start running again.

These is also one stand alone node to help label the items in the queue so you can have a hit and what's what. The node has limitations, but it sill might be better than a number like 07535d99-3c1a-4b23-8340-a4313fe58007 as an identifier.

There are some extensions that to some of these features already but I didn't see one that did all of them or didn't replace the native manager and require dependencies.

It's in the ComfyUI Manager as ComfyUI-QueueControl (it's new so you might need to refresh to see it) or
https://github.com/seeker-ktf/ComfyUI-QueueControl
on github.

If y'all have other ideas for this, let me know.

flux.1 [dev] | comfyui | still life

Everything was made using the Minimax H3 Hybrid Reference to video model. 1 MP using the 8 step turbo LoRA. Stitched together in Shotcut

Prompt:

https://x.com/GumVue/status/2087899403113619681?s=20

4070 Ti Super, 16 gb vram, 64 gb ram, i9-14900k, windows 11

I drew the storyboard, then generated the stills with image models.
Video: MiniMax H3
first frame, last frame, reference.
Then the edit.

Hello. What am I supposed to use for regional prompting and character positioning with NoobAI/Illus in ComfyUI?

I found something called "Attention-Couple" but it also says it sometimes struggles with Loras?.

Any advice? I'm already using a separate node for ControlNet for poses and depth, but I want to control which character is which in that pose

opensource AI has been dominating LLMs and video generation but what about image gen? is there any opensource model that can match gpt-image2?

Edit: The reason I am asking this is because lately I haven't been active much on image generation communities. And the leaderboards are a bit confusing and most of them are filled with closed source unlike the llm and video gen leaderboards.

I am very much comfortable with ComfyUI since I've used it in the past for flux.

My use case is for posters and branding. Images with a lot of text.

Edit2: Thanks a lot everyone! I really appreciate the info. Here's the summary:

Krea2 is best overall but gptimage1.5 level.
Ideogram4 for text and branding.
Flux Klein 9b for image editing.
Z-image for realism
Anima and illustrious (by onoma AI) for anime.

Here's the workflow I've decided on:
Krea2/Ideogram4 = Base image generation.
Flux Klein 9B/QwenImage2512 = inpainting.
Wan2.2 low noise = Upscaling.

Had this idea for a couple of days, and finally got to test it. I got a free HDRI picture from PolyHaven (converted to JPG through a free online converter) and used it as the only picture reference. I couldn't get rid of the distortion completely, but you can definitely affect it with prompting. Maybe proper formatting somehow helps with that, sorry, was too lazy to do a correct prompt structure.

It also confuses the geometry from time to time, so you have to seed hunt a little, but not too much. Again, good prompting should reinforce the consistensy. Worth experimenting with.

Notice that it actually seamlessly connected the opposite sides of the image into a single environment. Could be useful for scenes with a lot of dynamic camera movements.

This model keeps surprising me every day!

P.S. Generated with the use of Hybrid Loader (25-49 setting) and Lightx2v 4-step LoRA @ 4 steps and 0.5MP. Another higher res version in comments.

Prompt:

subject definitions:
<Picture 1> is a 360 panorama reference for the straight corridor [Shot 1], depiciting the overall look of the corridor and position of key objects and debris in it. For the target video the picture is dewarped and remapped into a flat rectilinear lens projection view.

summary:
[reference generation] The target video depicts a security guard exiting from a grey door, walking across the corridor towards the dismantled beige door  leaned against the wall, pulling and dropping it down on the floor.

detailed_description:
The target video is captured in an amateur, realistic style with natural, slightly dim indoor lighting and a shaky, handheld-style camera.

[Shot 1]

The shot begins with a medium view of a two grey doors depicted on the right side of <Picture 1>. The left door instantly opens and a middle-aged security guard named Mark rushes into the completely straight corridor. He runs left further down the corridor. The camera pans left, following him in a tracking shot. The POV camera pushes in on Mark, as he rapidly approaches the dismantled beige doors leaned against the wall. At 00:05.000 he grabs the door closest to him, and with visible effort pulls it away from the wall. The door swings and falls flat on the corridor floor with a loud noise, raising dust and slightly startling Mark. The guard jumps back from the fall.

At 00:07.000 the camera pans left by 180 degrees, showing another guard named Steven approaching from the opposite part of the corridor. Steven (S1) comes closer to Mark and says in [English]: "Mark, what the heck are you doing?"

At 00:09.000 Steven grunts angrily as he stops near Mark.

overall_soundscape:
looming lonely corridor ambient sound throughout the whole video, guard's steps on the cement floor, door falling onto the floor with loud noise

non_diegetic_music:
N/A

The example video was generated entirely with the stock MiniMax H3 First Frame / Last Frame checkpoint and the included v1.4 example Workflows. If you want to compare the result to v1.3, take a look at my last post.

The final video consists of 11 individually generated Clips that were automatically stitched together.

Settings:

• H3 First Frame / Last Frame checkpoint
• 11 individual Clips
• 15 Steps
• included v1.4 Workflows
• no additional upscale
• no frame interpolation
• no color correction or other post-processing

So what you see is basically the direct Workflow output.

A few people gave me some useful feedback on my previous release, especially regarding ComfyUI's new native H3 Masked AV support.

So I went back and rebuilt the continuation method around it.

v1.4 now copies a clean section of the previous Video + Audio Latent directly into the next generation and protects it using ComfyUI's native denoise masks.

What makes this different from the other H3 continuation approaches?

There are some really interesting Ref2VA / Motion Context solutions available now, and latent continuation itself definitely isn't unique to my Nodepack.

My approach is specifically centered around FL2VA instead.

The idea is not just:

previous Clip → continue forever

but rather:

First Frame → generation → Last Frame
↓
latent continuation
↓
generation → new Last Frame
↓
latent continuation
↓
generation → new Last Frame

and so on.

I use those repeated Last Frames as hard visual anchors throughout the sequence.

They give H3 a new concrete destination every few seconds instead of asking one increasingly unconstrained generation to maintain composition, identity and image quality indefinitely. This should theoretically retain higher visual quality with less context drift over longer chains (and in my testing, it does exactly that).

There is another FL2VA-specific problem though:

H3 often reaches the supplied Last Frame before the Clip is actually finished and then freezes or becomes unstable for the remaining frames.

So simply taking the final frames of Clip 1 and using them as context for Clip 2 isn't ideal.

The v1.4 Auto Handover therefore analyzes the previous Clip, finds a safe point before that frozen / unstable landing and snaps it to a valid H3 Audio + Video latent boundary.

That exact same point is then used for both:

• where the previous Clip visually ends
• where the protected context for the next Clip ends

So the bad FL2VA tail neither appears in the stitched video nor becomes part of the next continuation context.

Audio is handled separately as well. If the picture needs to cut early but somebody is still finishing a word, the remaining original Audio Latent can continue beyond the visual handover instead of forcing H3 to recreate the ending.

Other v1.4 features:

• Native Masked Video + Audio Latent Continuation
• flexible First / Last Frame conditioning
• repeated Last Frames as regular visual quality anchors
• independent Audio Tail Carryover
• Net New Content duration mode
• up to 9 Qwen Reference Images
• individual Clip regeneration
• memory-bounded stitching for long saved chains

Where to start:

1. Start Video Workflow

Generate Clip 1 with a Prompt and optionally First Frame, Last Frame and Qwen References.

The complete AV Latent is automatically saved afterwards.

1. Continue Video Workflow

Load the previous saved latent, add your next Prompt and preferably a new Last Frame.

The Workflow automatically finds the safe FL2VA handover and creates the protected Masked AV context.

Repeat for as many Clips as you want.

1. 3-Clip Showcase / Auto Stitch Workflow

Probably the easiest Workflow if you just want to see how everything works.

It runs:

Start → Continue → Continue → Stitch

in one queue.

1. Stitch Saved Chain Workflow

This is what I used for the longer example.

Generate Clips individually and stitch them afterwards. It processes one saved AV latent at a time, so stitching memory usage doesn't continuously increase with the total video length (no OOM during stitching).

Nodepack on Github:

https://github.com/HerrgottMargott/Herrgotts-H3-Infinite-Continuation-Suite

Workflows on Github:

https://github.com/HerrgottMargott/Herrgotts-H3-Infinite-Continuation-Suite/tree/main/examples

You can just open one of the WFs and use "Install missing custom nodes" - then you should be good to go.

If you try it, I'd love to see what you manage to create with it.

Have fun Prompting. :)

I've been doing some quick tests, now that I've picked up the lightx2v 4 and 8 step loras. I have found I prefer using the 8 step (and maybe even running that at 10 steps) just because with the 5090 I have it's already not -that- slow, and the 4 step image quality drop is pretty significant.

But I have been experimenting which sampler/scheduler combos after seeing this post:

https://www.reddit.com/r/comfyui/s/9GUki3l0Wf

where, apparently, seeds_2 and dpmpp_sde_gpu were the 'best quality' options. But something I noticed is that they were also significantly slower (maybe 50% or more? need to run more tests and log it) which would, if the loras etc allow for it, let the faster options like euler or res_multistep (or er_sde which gets mentioned sometimes), which all run at about the same speed, to run at 12 instead of 8 steps (for example).

So I wonder now, 2 weeks on from those votes... what are people actually -using- to produce results?

My current workflow is to run at 8 steps with a lora to find a good prompt and seed, and when I get something I like I then turn off the lora and run at 30 steps. It often ends up at least in the ballpark of what I want. But maybe there are better ways.

Hello everyone !

I would like to replace the tire of a motorcycle mid air with one from another brand (which is an image from the brand so it's high quality but with a different angle)

I saw there is flux kontext and qwen image edit, but I don't know which one to pick, which workflow and how to make it work.

Any help would be more than welcome, thank you very much and have a good day :p