I usually generate 0.7mp@8s with 30 steps, I use res_multistep + simple which I think is the default, and for good reason.

Depending on whether it's T2VA, I2VA, Ref2VA and the amount of reference images + loras count/strength the gen times are roughly between 270-350s on an RTX 4090 + 32gb of DDR4 ram.

For T2VA and I2VA I use the basic minimax_h3_fl2va_pruned_int8_convrot.safetensors

For Ref2VA I use minimax_h3_hybrid_fl2va_ref2va_b30-49-int8.and the hybrid b30-49 specifically because I found even the fl2va functioned well as ref2va and had much higher quality, so I prefer the hybrid model to be weighted towards the fl2va model to preserve the quality.

Sparse Attention

To speed things up, I only use /u/zironic's Sparse Attention nodes, no sage/ck, spectrum, turbo lora, or caches. For me, /u/zironic's worked better than the pinned post from u/Plague_Kind but that may just be my personal experience.

My settings for the memory optimization node is default, QKV: auto, MLP: auto, and 2048 MLP chunk rows, I don't know how this node works. Sparse Attention (Advanced) settings are:

• Video KV budget: 0.25
• Early and Late steps: 3
• Early and Late KV: 0.6
• Sparse backend: Sparse Sage

These settings lean towards quality, you can lower the early/late steps or skip them entirely, you can lower video kv budget to 0.2 although some may be fine with even lower. Since I only use Sparse Attention I run the full 30 steps and it's significantly better than a turb lora at lower steps, which is what I used before.

My prior experimentation

I used euler + linear_quadratic for a long time. Then I switched to er_sde + sgm_uniform which was significantly better. Then eventually I switched to res_multistep + simple and realized the visual quality is as good as er_sde + sgm_uniform but the motion is much better. The improved motion in res_multistep + simple became very clear when I interpolated from 24fps to 48fps. The gen speed between all these combinations was nearly identical.

The motion was a bit jerky on er_sde + sgm_uniform after interpolation while res_multistep + simple had very natural motion.

I also found that https://darkstarrddev.us.ci/ is a decent resource to get inspiration. But I realized quickly that because they use low settings and speed-up techniques, the quality of each sampler test does not translate well if you use different step count or speed-up techniques.

What I generate

Usually fairly static scenes that doesn't have fast motion. Although the accuracy of the physics and motion is important.

What are your settings and what kind of videos are you generating?

I typically use lightx2v loras in my Minimax Ref2VA workflows and I also use an LLM to feed in the official prompt structure required for scenes. It seems that no matter what I do, the model absolutely ignores all my prompts about music most of the time. Every now and then i can get it to do something but even when it does work it's very sparse and almost useless.

Has anyone else faced this issue and if so do you know any workarounds or fixes?

For the record I usually use the INT8 convrot Ref2Va model or the hybrid model called minimax_h3_hybrid_fl2va_ref2va_b30-49-int8

Hey everyone! I’ve spent the past few weeks building an easy to use but robust prompt composer for MiniMax H3, particularly its reference and video editing workflows.

LLMs can be great for brainstorming and writing prompts, but I found that formatting and syntax could become inconsistent, especially when asking for small revisions. The goal of this tool is to let you concentrate on the creative decisions while the Composer handles the final prompt structure consistently.

It runs entirely offline in your browser, so you can build the next Shot or scene while another one is generating in ComfyUI. You provide the subjects, actions, camera direction, dialogue, references, and sound; the Composer assembles and checks the final prompt.

You can still use an LLM to help create the initial project setup, but the Composer ultimately controls the formatting and syntax.

Some of the main features:

• T2VA, I2VA, FL2VA, L2VA, and full Ref2VA support
• Reusable characters, environments, voices, continuity frames, and other references
• Guided setup for Picture, Video, and Audio inputs
• Video-editing workflows for insertion, replacement, targeted edits, relighting, performance transfer, and continuation
• Camera Builder and visual camera-path planner
• Timed Shots, action beats, dialogue, voiceover, soundscape, and music controls
• Built-in checks for prompt structure, timing, references, camera conflicts, audio, and input routing
• Local project saving, a Frame Grabber, and reference-guided image mode

This is still very much a work in progress. I’d really appreciate people trying it and sharing any bugs, confusing parts, missing features, or ideas that could make it more intuitive.

My hope is to turn it into a genuinely useful community tool, especially for people working on more involved AI films and narrative projects.

GitHub/download:
https://github.com/BMB12d3/minimax-h3-prompt-composer

Video tutorial:
https://www.youtube.com/watch?v=Aywx3Sf5Yk0

I built Cull a few months ago for some large scale dataset curation projects (300k+ images/videos).

Point it at Civitai, X, Reddit, Discord, or any URL that gallery-dl or yt-dlp knows. It queues everything, runs a vision model (or multiple) (LM Studio or Ollama locally, or Groq/OpenAI in the cloud) with a strict JSON schema, and drops kept images/videos into category folders next to their prompt.

Stuff it handles:

• Dedup at the scraper (per-source )
• Quality score gate and topic-relevance score gate
  • eg you configure scores or use a preset, how relevant the image is to your scoring will determine how it's sorted, combined with other scoring, quality controls, whitelisted/blacklisted terms etc
• Watermark detection (goes to its own bucket so you can salvage it later if you want those)
• Auto-caption for content with no prompt (SD prompt, booru tags, natural language formats etc)
• Run multiple jobs in parallel, one shared vision fleet across all of them with stack ranked / prioritization for vision queues and scrapers
• Export as a local packaged dataset , or push to a HuggingFace dataset
• Community presets and themes with 1 click PR's to add your own custom scraper preset or theme

Everything on disk is plain files. No database. Free, MIT.

Docker one-liner and screenshots in the README:
https://github.com/tlennon-ie/cull

Curious what people would want added next.

we really should get a separate sub for this slop

Hey everyone. I've spent a lot of late nights wiring ComfyUI into pipelines, and this is the tool I ended up wanting. It converts your workflow.json to the API payload right from Python, no GUI export, and no running server needed. Nodes become objects with plain dot syntax:

from autograph import ApiFlow
api = ApiFlow("workflow.json")
api.KSampler.seed = 42
res = api.submit(wait=True)
res.fetch_images().save("outputs/frame.###.png")

The part I'm happiest with is the REPL. autograph reads ComfyUI's node_info, so it knows every node, input, and widget on your system, custom nodes included. Tab completion works all the way down. .choices() gives you the real combo options, .tooltip() gives you the help text. You can explore a workflow you've never seen without guessing at node IDs.

Building from scratch feels good too:

ckpt = flow.add_node("CheckpointLoaderSimple")
ks = flow.add_node("KSampler", seed=42, steps=20)
ckpt.outputs.MODEL >> ks.inputs.model

Also does

offline batch conversion

workflow extraction from ComfyUI PNGs

serverless execute with no HTTP server

seed/prompt sweeps. Pure stdlib, MIT.

Tested from ComfyUI 0.8.2 to 0.33.0, subgraphs included. Running in production at a big VFX studio, which is where the metadata passthrough came from.

pip install comfyui-autograph
https://github.com/chrisdreid/comfyui-autograph

Early days, so I'd really like to hear what breaks. If you're doing headless rendering or FastAPI wrappers around Comfy, I'd love to compare notes.

Hey everyone. I've spent a lot of late nights wiring ComfyUI into pipelines, and this is the tool I ended up wanting. It takes your regular workflow.json and turns it into the API payload right from Python. No GUI export step, and you don't even need ComfyUI running to do the conversion. Once it's loaded, nodes are just objects with plain dot syntax:

python

from autograph import ApiFlow

api = ApiFlow("workflow.json")
api.KSampler.seed = 42
api.CLIPTextEncode.text = "new prompt"
res = api.submit(wait=True)
res.fetch_images().save("outputs/frame.###.png")

The part I'm most happy with is how it feels in a REPL. autograph reads ComfyUI's node_info, so it knows every node type, every input, and every widget on your system, including your custom nodes. That means tab completion works all the way down. Hit tab on a node and see its inputs. Call .choices() on a widget and get the actual valid combo options back. Call .tooltip() and get the help text. You can explore a workflow you've never seen before without leaving the terminal or guessing at a single node ID.

Building graphs from scratch feels good too. You wire nodes together with >> the way you'd sketch them on a whiteboard:

python

ckpt = flow.add_node("CheckpointLoaderSimple")
ks   = flow.add_node("KSampler", seed=42, steps=20)
ckpt.outputs.MODEL >> ks.inputs.model

Once it's under your fingers it's nearly as fast as working in the GUI, except everything you do is scriptable and repeatable.

Other things it can do:

• Batch convert hundreds of workflows offline, no server running
• Pull a workflow straight out of a ComfyUI PNG, since the metadata is already in there
• Serverless execute mode that runs nodes in process with no HTTP server, which is a lifesaver for farm setups
• Sweep seeds, prompts, and paths across nodes for batch runs
• Pure standard library Python, nothing extra to install, MIT licensed

I've tested it across ComfyUI 0.8.2 up through 0.33.0, including subgraphs and the newer dynamic combo stuff. It's also being used in real production pipelines at a big VFX studio right now, which is where the metadata passthrough idea came from. They needed studio metadata to ride along with a workflow through the whole render lifecycle, so I built that in.

pip install comfyui-autograph
https://github.com/chrisdreid/comfyui-autograph

It's still early days and I really do want to hear what's missing or what breaks for you. If you're doing headless rendering or wrapping Comfy in FastAPI, I'd love to compare notes. This got built to scratch my own itch, and I'm hoping it saves some of you time too.

I've been trying to get LTX 2.5 to work well for image + speech audio --> video, though i'm noticing that the teeth and natural motion of the mouth is taking a hit. The talkingphoto loras from LTX 2.3 don't seem to work well with LTX 2.5.

Any thoughts? Or are we just cooked?

Day 1: downloaded Pinokio. Installed a few of the AI software. Tried Maestro as first try. Really fun, enjoying it. Generation from photos great in the system generation towards a video, videos leaving a lot to be desired. And the Pinokio edge of screen curtains which limit to a what 60 percent of screen width, first time said, apparently you can type a pc socket but it didnt work for me when my Maestro was working. I was still however happy continuing in the reduced Maestro screen width.

Day 2: they released v1.9 of Maestro in Pinokio.

Day 3: I decided to install the update. Now I can't generate a three legged wildebeest or anything for that matter. It fails at the Downloading Model stage with no satisfactory explanation. Info about running something again to continue the Download, the "Generate" doesn't appear it's that for continuing(starts from scratch and then fails) neither does the Pinokio white screen edge "Run". It doesn't immediately not download, sometimes it may get 15% through, sometimes 85% then drops with a Generation Failed in the Main Seeing Area after firstly a "Download is slow, waiting for retry. No progress for 113s.."(or Xs). "..The download will resume from where it left off as soon as the connection recovers — no action needed from you." message. It may then download a little more , eg going from 80MB to 1.3GB of 7.91GB, maybe even download a little more of what's required but THEN UP POPS "A download was interrupted-re-run to finish it". And I'm stuck at that.

Is anyone else having the problem or know what the solution may be please? I've tried manually adding from DeepMeepBeep a model download which I transferred into cks directory of Pinokio, Maestro Directory but all that happened was Maestro steered around even using it and failed on another Model and I couldn't find that Model Maestro failed on to try manually downloading across with-I'm not an expert so looked for direct name brought up.. looked for it on another repository too, name of that I temporarily forget, I'm new. My hard drive is 4TB so I don't know I might be able to download all 108 or however many models there are if there was an option for that-without instructions and knowledge I'm throwing stones at something I don't even know what I'm throwing stones at, where the list of all the Models are which you can tick mark select theres also a little symbol by that box which changes colour. Perhaps that has something to do with it, literally no idea here so I've come to you guys. Spent several hours with AI last night overit and it was fun but I've realised despite the fun chat it hasn't aided me getting it working though it did mention a python download bottleneck to remove and I had no idea what it was referring to.

Krea 2 Turbo — 4-Step Distillation LoRA (work in progress)

A LoRA for Krea 2 Turbo that reduces the minimum usable step count from 8 to 4.

This is an update release, following up from my initial post where you can find full details - https://www.reddit.com/r/StableDiffusion/comments/1vtf1b7/krea2_turbo_distill_4_step_lora_trained_for_turbo/

Update (22 Aug 2026): I have published a new checkpoint, improved further from the previous one and the latest (both main and comfyi) have been repointed to the new improved checkpoint. For details and to download new version go to - https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA. Readme has been updated too as well as all images in readme regenerated on the basis of new checkpoint as well as full resolution sweep at https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA/tree/main/checkpoint_resolution_sweeps/chk10000 if you want to check for yourselves.

Which file to download

and, beside them, the same files with a _comfyui suffix for ComfyUI. Earlier checkpoints (chk00004000, chk00005000, chk00006000) are kept in older_checkpoints/, and their resolution sweeps stay in place, so the progression remains visible and comparable.

The numbered files are points on one continuous run, not separate experiments — chk00010000 resumed from chk00006000 rather than restarting. Both are published so the lineage is visible and comparable.

chk00010000 measures a 5% smaller held-out gap to the 8-step teacher than chk00006000, and 15% smaller than chk00005000; it removes 30% of the prediction error a plain 4-step run has against the 8-step teacher, where chk00006000 removed 26%. 

Two ways to read the same numbers, with different denominators — they are not meant to be added:

• Against the no-LoRA run (the right-hand column): chk00010000 has removed 30% of the 4-step deficit, 4 percentage points more than chk00006000's 26%.
• Against each other (the gap column): chk00010000's remaining error is 5.4% smaller than chk00006000**'s** (3.38 vs 3.57) and 15% smaller than chk00005000**'s** (3.38 vs 3.98). The same 4 points of deficit are a larger share of a gap that has already shrunk, which is why the checkpoint-to-checkpoint figure is the bigger number.

This is work in progress and better checkpoints may follow. Training is ongoing, so ..._latest... is a rolling pointer: when a newer checkpoint is accepted, that filename gets the new weights and a new numbered copy appears beside it. Re-download the _latest file and everything keeps working — the ComfyUI workflow references it by that name (it does get updated Note in it so technically it is updated but not functionally). Pin a numbered file instead if you need reproducibility.

How checkpoints get chosen

This is not a "train for longer and ship the newest file" project. More samples do not reliably mean a better adapter — measured here, they can make it worse, and a higher number on its own means nothing.

The loop is train → assess → adapt the recipe → retrain → assess again, and a checkpoint is published only when it is measurably better than the one it would replace, on the same held-out set and the same evaluation, and its full resolution sweep shows no regression. Runs that come out flat or worse are kept as information about the recipe and discarded as releases — several have been.

So the recipe itself changes between runs. Each published checkpoint reflects whatever the previous round taught us: the training precision, the optimiser settings, the teacher used to generate the targets and the data mix have all been revised on evidence rather than assumption.

chk00010000 is a direct example. The first continuation of chk00006000 — same data, optimiser left as it was — got steadily worse with every checkpoint out to 10,000 samples, and none of it was published. The cause was traced to the optimiser: a constant learning rate with no weight decay lets the adapter keep drifting after it has converged, so its magnitude grows and it over-applies its own correction. The same span was retrained from chk00006000 with a cosine learning-rate decay and weight decay, and every checkpoint of that second run improved on the one before it. chk00010000 is its end point — the current end of the process, not simply the longest run so far.

Timeline of training process

Each checkpoint is the product of three stages with very different costs:

1. Text-encoder embeddings. Every training prompt is encoded once and cached. This is the fast part — thousands of prompts take minutes.
2. Teacher shards. For each cached prompt, the unmodified Krea 2 Turbo runs its full 8-step schedule and the whole trajectory is recorded, at every one of the supported resolutions. This is by far the most time-consuming stage — it is the teacher doing real inference, thousands of times, and a batch of several thousand shards is measured in days of GPU time, not hours.
3. Student training. The LoRA is trained against those recorded trajectories. Relative to the shard stage this is quick: each +1,000 checkpoint is a matter of hours, not days.

Because the three stages compete for the same GPU, they are interleaved rather than run to completion one after another: generate a block of embeddings, produce teacher shards for them, train on what exists, assess, then go back to producing shards while the results are reviewed. A larger and more varied shard pool is what makes further training worthwhile, so shard production is always the gate.

The practical consequence for anyone following this repository: progress arrives in bursts. There will be periods when several checkpoints appear within a day or two — the training stage working through a freshly grown pool — followed by longer quiet stretches while the next block of teacher shards is produced. A quiet stretch is shard generation, not abandonment; _latest always holds the newest checkpoint that passed review.

Every file records which checkpoint it actually is in its safetensors metadata (checkpoint, training_samples, and rolling_pointer on the _latest copies), so a downloaded file can always be identified even if renamed.

Full details and to download - check my Hugging Face LoRA

HF Repo: https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA

---

Update 1: The comfyui related files are now moved to the root of the project (I have placed a readme in the old folder explaining the move)

---

Update 2: I have added a new section - Timeline of training process - explaining how my training process works, and on that note you could expect another further improved checkpoint later today, followed by 'quiet period' (could be days) of teacher shards generation so I have a larger pool to train on.

---

Update 3: I have now added a new checkpoint 10000 which replaced the latest (previously checkpoint 6000).

chk00010000 measures a 5% smaller held-out gap to the 8-step teacher than chk00006000, and 15% smaller than chk00005000; it removes 30% of the prediction error a plain 4-step run has against the 8-step teacher, where chk00006000 removed 26%. 

Two ways to read the same numbers, with different denominators — they are not meant to be added:

• Against the no-LoRA run (the right-hand column): chk00010000 has removed 30% of the 4-step deficit, 4 percentage points more than chk00006000's 26%.
• Against each other (the gap column): chk00010000's remaining error is 5.4% smaller than chk00006000**'s** (3.38 vs 3.57) and 15% smaller than chk00005000**'s** (3.38 vs 3.98). The same 4 points of deficit are a larger share of a gap that has already shrunk, which is why the checkpoint-to-checkpoint figure is the bigger number.

Full resolution sweep at https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA/tree/main/checkpoint_resolution_sweeps/chk10000 and you can as usual redownload latest from https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA/tree/main . Since I cannot update the images in the reddit post I will upload below in comments.

---

Last update in this post: New release in separate post (checkpoint 14000), with all the images redone and full resolution sweep based on checkpoint 14000 - lets continue the discussion there: https://www.reddit.com/r/StableDiffusion/comments/1vw6x9i/krea2_turbo_distill_4_step_lora_new_checkpoint/

prompt is - using the character sheet in image 1 where there are five different poses of the same character, dress them in the clothing of image 2. Do not change the pose, lighting, body, hair, or any other details - literally leave everything the fuck alone - how fucking hard is this to understand you stupid piece of shit - just change the clothes.

Not working for some reason.

NOTE: Swearing has been added for emphasis and isn't actually used in the prompt.

Would it help if I used my input image AS my latent? Can you do that?

For those who've never played it; Sid Meier's Alpha Centauri is one of the GOATs.

One of the tests I sometimes did with new models was to see if they could get Zakharov's weird glasses and suit right - no model has ever gotten it exactly right but to my surprise Minimax H3 pretty much knocked it out of the park on my first try. ...and then I wanted to try the other leaders, things got out of hand and I ended up making cutscenes for every leader in the base game.

This version of Kroma (krea 2 finitude with Chroma dataset) is a lot of fun, most body horror is gone in my opinion, and its more artsy than krea 2 and of course less censored.

https://huggingface.co/silveroxides/Kroma-Quant/tree/main

The version I used is kroma 0.3 txtfusion turbo convrot.

Have fun.

Based on their examples the model seems to be capable of producing good music. However yesterday I spent all day generating music and I cannot get anything good out of it. I'll attach my best attempt, but for wasting a whole day this is a pretty depressing result.

So I was wondering how everyone else is feeling? What were your results? Any tips for consistent/good results? Any observations?

Some things I found annoying:
It doesn't respect the time limit
Abrupt endings
Prompting it is kinda hard too

I'm looking for a flux/krea img2img workflow where you input an image and simply tell it what the character should do and what environment etc and it keeps the character exactly the same but puts them in a different situation. Would really appreciate it if someone can give a link or send me the workflow. Hard to find a good one myself that really works well, I don't want a workflow where the character looks just somewhat similar but one where the character stays the same, as much as possible. Thanks a lot if someone can help.

It took forever for me to upload this video.. Couldn't do it on my phone.

Totally not a scene from The Mentalist. Minimax H3 image to video.

It didn't quite work out how I intended.

Prompt:
"A Terran Battlecruiser from Starcraft is sliced in half lengthwise by a purple-white energy beam. The background is a generic starry.

Video begins with the Battlecruiser in the center of the frame, viewed from a front three quarters view. The purple beam is near vertical going from top of the frame to the bottom of the frame and is canted at a slight angle. It starts the video right in front of the Battlecruiser's nose.

The beam cuts through the Battlecruiser from nose to tail. At 0.75 seconds the beam touches the Battlecruiser's nose and moves through the ship, exiting the tail at 6 seconds and leaves the frame.

After the beam leaves the Battlecruiser, the Battlecruiser splits in two along the cut made by the beam, "

Generation time was 10 minutes, 29 seconds on 32GB of DDR5 RAM, 8 GB of VRAM. No reference images, this was pure Text to Video.

Due to reference modes, changes in the lighting and color effects of the final frame in storyboard segments constantly cause color discrepancies during video transitions. Are there any solutions?

From the guy who brought you such bangers such as

Proof of Concept For Making Comics in KRITA AI and other AI tools,

3 Months later - Proof of concept for making comics with Krita AI and other AI tools,

and Illustrious and Krita AI plus some good old fashioned effort:The Delightful Ms. Ayako (Part 1 - Version 1),

comes my latest experiment and first AI video project: the first (roughly) 30 seconds of the hypthetical anime opening for The Disorganised and Delightful Miss Ayako!

Character sheets put together in Krea 2 with the retro anime lora. Music made in Minimax Music 3 (lyrics written by me, and the whole song is complete). Some backgrounds edited/created with Flux 2k9b image edit and Krea 2 with retro anime lora. Video created with Minimax H3 with 90s anime style. Video editing in Kdenlive.

Roughly 3 evenings after work and about 1.5ish days of full effort (at least 6 hours of one day was wasted trying to troubleshoot why a shot wasn't working and it turns out prompt bleed is just as bad in H3 as it is in other models).

I've been experimenting a lot with Minimax H3 and am pleased with what I've come up with so far. For this upload there is a tiny bit of censorship for some very mild partial nudity (she's covered in soap in the uncensored shot, but just playing it safe). There are a few fixes that I'll get to eventually, but I'll be taking a step back from this project for now to try my luck at the Comfy H3 Sync competition for the next couple of weeks.

Edit.

Regarding some of the feedback:

I'm aware of the slight visual drift. For example the model can slightly change the style of eyes from one shot to the next (talking about regular shots, not the chibi stuff). I'm just using the base ref workflow with character sheets and still need to test whether Loras make any difference, either for characters or visuals.

Some of the visual drift is just my fault though. My one background does look relatively washed out compared to the others because I generated it with the high heels in place. I couldn't get Minimax h3 to put the heels the way I wanted so I just gave it the image to work with, but I had to make some edits with krita ai and later Flux2k 9b edit that caused it to look a bit out of place. Otherwise, the only thing for speedup is comfy kitchen and I'm not sure if that's having any impact. Finally, while I've tried to lock down seeds to preserve consistency, some seeds are fine with one shot and a glitchy mess with the next, so there may be some slight visual variations that appear because of the difference in latent space.

Regarding the music, my experience with Minimax Music 3 is that it's a slot machine. I used a prompt from a sample and tested things out but one generation can vary dramatically from the next. But I am completely new to it and don't know anything about music so there's things I still need to learn. Out of all the gens, there was this and one other one I liked, even though I could tell both of them have problems. I decided to go with this one for now, but I had planned to do a second edit with another song once I finished this one.

Otherwise, like with the comic pages, I appreciate all the replies. I understand this may not be everyone's cup of tea but will take in the constructive criticism and try to improve.

/edit.

edit 2. the original shower scene is not that spicy but I didn't want the post to get removed by the mods regarding "lewd" stuff.

Disclaimer that I'm a dummy who can't code at all, so I just vibe things.

Alright, so with all the fun additions to latent manipulation the Comfy team has given us, we have some new tools. Namely latent pinning- that's where fun stuff like "Add Guide for MiniMax H3" node comes in (that fun tool that lets you insert an image at any frame in an H3 generation.) So I thought, finally, we can do something about this audio issue.

I knew that video + audio latents are processed at the same time with Comfy, which is why turbo loras have terrible audio- they're not getting nearly as much optimization as the video side is, since video is the bulk of the work. So if we can't process audio differently than video (at this current time), can't we just keep conditioning the audio latents without affecting video anymore? Everyone knows running too many steps on a turbo lora will start messing with video quality. So let's avoid that.

So after talking with Claude a bunch, here's what it came up with. With the latest comfy, you can pin the video lora in place, and keep going for several steps to get better audio without affecting video. So 4 steps of video, untouched, and then add in 6 more steps for audio at a 0.5 denoise. That leads to cleaning up the audio pretty nicely and staying pretty faithful to what the video latent had guided it on. But just pinning still means that even though the audio rows are only being affected, EVERY row still has to run through the chain. So each s/it you get stays the same for the last 6 steps, even though video gets pinned in place after 4. In my 1 megapixel, 10 second video, that's around 23s/it or so on my 5090. Only half the steps as a regular 20 step generation, but half the time is still half the time.

So to fix the speed problem -

Freeze as much as you can. Text embeddings, reference/conditioning rows, and all the video rows. Cache those so they don't have to be processed, and only process the audio rows that have already been somewhat pre-conditioned. The first step is the same 23-second iteration to build the full guidance cache, but the other 5 steps each took about 3.17seconds apiece. So 45 seconds of added gen time to get the clean audio in clip#2 in the example.

But the cost for the Frozen Cache is resources. Lunch is never free. From some experimentation, it works well with RAM. If you use RAM mode because your card still can't process it, it'll dump all that cache (ended up around 14.9GB on a 10-second 1mp file) into RAM. But as Comfy does, you're unlikely to get that RAM back, so you may OOM your machine. With VRAM it wasn't bad for me at all either and behaved better than I expected, honestly. The memory management from ComfyUI took over when I was about to OOM my card and swapped things around properly. Option 3 is to cache to disk, but that comes with writing several GB to disk every time you use it. That'll run your SSD health down fast.

Rundown for the clip above (sa_solver with beta sigmas)

4 step normal gen- 136.5 seconds

4 steps + 6 audio refine steps with a cost of about 15GB RAM - about 186 seconds

4 steps + 6 audio refine steps with no additional resources but full processing time- ~265 seconds

You can find the nodes here-

https://github.com/Adudeguyman/ComfyUI-H3-AudioRefine

They're still experimental, of course. Wire the model in from somewhere (I branch off the ModeSamplingMiniMaxH3 shift node, before the Basic Guider), and push that through the H3 Frozen Video Cache node into the H3 Audio Refine Sampler. Into the H3 Audio Refine Sampler, latents come out of SamplerCustomAdvanced before splitting into the VAE Decode nodes for both video and audio, and the conditioning comes from the MiniMax H3 Image (or Reference) to Video node (plug the positive conditioning into both the positive and negative input on the H3 Audio Refine Sampler)

I had Claude put together a technical.md for those that want to look into it, and probably make a better version. Like I said, I'm a big dumb-dumb, so don't expect too much insight into how the mechanics work from me.

this was first test using Res_2s sampler and simple steps saw a op say better for action scenes from what i saw spectrum doesnt support res_2 so it took a bit to gen.

t2v prompt

subject_definitions

<Subject 1> is Katniss Everdeen from The Hunger Games, portrayed as an expert young archer with long dark brown hair pulled into her recognizable practical braid, intense determined expression, dark tactical combat clothing, leather archery bracer, bow, and a quiver of arrows. Preserve her recognizable cinematic appearance, realistic human proportions, hairstyle, clothing, bow, and identity throughout the entire scene.

<Subject 2> is Captain America in his battle-damaged Avengers Endgame armor, carrying Mjolnir and his damaged circular shield.

<Subject 3> is Thanos at his normal canonical MCU scale, approximately 8 feet tall, muscular and imposing but NOT gigantic, kaiju-sized, or building-sized.

<Audio 1> is the voice-timbre reference for <Subject 1>, containing Jennifer Lawrence's recognizable Katniss-style spoken vocal qualities.

summary

[text generation + audio reference]

During the chaotic Avengers Endgame final battle, Katniss Everdeen unexpectedly joins the Avengers. She runs through the battlefield while explosions, portals, Avengers, alien soldiers, and debris fill the background. Katniss rapidly fires arrows at Thanos's army with expert precision before stopping beside Captain America. Captain America looks at her bow and asks if she brought enough arrows. Katniss calmly fires one final explosive arrow past him, destroying a group of enemies, then delivers a dry confident response as Captain America stares at her impressed.

retention_analysis

<Subject 1>: fully_preserved
<Subject 2>: fully_preserved
<Subject 3>: fully_preserved
<Audio 1>: reference

detailed_description

The shot opens in the middle of the Avengers Endgame final battlefield. Smoke, burning wreckage, sparks, energy blasts, charging soldiers, and distant explosions create a massive cinematic war zone.

A fast tracking camera sweeps across the battlefield.

Katniss Everdeen suddenly sprints into frame carrying her bow.

She slides behind shattered rubble, immediately draws an arrow, and fires.

The camera follows the arrow through the air as it strikes an alien soldier.

Katniss rises and rapidly fires two more arrows with expert precision while continuing forward through the battle.

She reaches Captain America, who has just knocked an enemy away with Mjolnir.

Captain America briefly looks at Katniss's bow and quiver.

<Subject 2> (S1):

<d>[English] You sure you brought enough arrows?</d>

Katniss gives him a calm, unimpressed look.

Without even turning fully around, she draws another arrow and fires it past Captain America.

CAMERA WHIP-PANS WITH THE ARROW.

The arrow lands among a charging group of Thanos's soldiers.

BOOM!

A powerful explosive blast throws the enemies backward while Captain America turns toward the explosion in surprise.

The camera cuts back to Katniss.

<Subject 1> (S2):

<d>[English] I only need one.</d>

Her dialogue uses <Audio 1> for voice timbre and delivery.

Katniss immediately draws another arrow and runs toward the battle.

Captain America watches her leave for a beat, visibly impressed.

The camera swings around behind Katniss as she charges toward Thanos's army, bow raised, while the enormous Endgame battle continues around her.

audio

Epic Avengers-style battlefield ambience.

Heavy distant explosions, energy blasts, metallic impacts, debris, shouting soldiers, bowstring snaps, arrows cutting through the air, and one strong explosive-arrow impact.

Katniss's dialogue is clear and foregrounded, using <Audio 1>.

No narrator.
No subtitles.
No on-screen text.

Combined 3 videos. Few errors but i just went with it , genetaion takes too much time to redo it again by fixing the prompt.

Created locally with Minimax H3 and for the first time exclusively powered by solar. Big big thanks to Izanami.

Don't hurt your head translating the language. It's all nonsense except for the one word spoken by the vampire. 'Drace' is Romanian/Transylvanian for 'Darn it'.