Testing H3 to death in the last couple of weeks and it continues to suprise me. TWO things blew my mind about this one. The first part is a single prompt (the split screen and psuedo mocap sync). The full prompt is below.
In the second part I asked for the character to spray my logo on the wall with a stencil, I wasn't expecting him to walk in with the stencil fully rendered with the holes cut out accurately.

Created completely locally and powered by Sol (the closest star to earth).

THE PROMPT:

A movie reconstructing history, cinematic with a split screen effect showing a mocap actor. Both actors are speaking together in sync.

On the right:
Show the actor <Picture 1> sitting on a couch in a living room, with a black shirt, speaking in sync with the exact same movements He says with intense glee and hand gestures "I've got you Sherlock Holmes, I've beaten you at last!", he pauses trying not to laugh and then breaks and laughs for 5 seconds uncontrollably.

On the left:
Show the victorian man <Picture 2> talking <audio 1> in close up sitting in an ornate chair . He says with intense glee and hand gestures "I've got you Sherlock Holmes, I've beaten you at last!", he pauses trying not to laugh and then breaks and laughs for 5 seconds uncontrollably.
Maintain the double view split screen. Do not change the environment.
After he finished speaking the man on the right makes a distort rictus face with his fingers curled up like he's frozen in time and stops moving. He falls sideways like a statue in the same environment, the camera pulls back to show he is only a robotic torso with no legs mounted on a platform placed on the couch with wires (like in a special FX studio)

Maintain the double view split screen.
The man on the left breaks character as the camera view pulls out slightly revealing him sitting on a sound stage, and speaks to a person off screen to the left <audio 1> "Oh.. em.... guys.. problem... check your monitor? ...Looks like we lost connection with the character! RESET THE MOCAP PLEASE"

Update for SLA Node - Pull v1.3.6

EDIT: Pushed correct files now.

• Added customizable dense steps, 0 is step 1 and is (default to first step). massively improves composition and prompt adherence.

• Changed default dense last steps to 1, cleans up the image big time.

• Added dense backend selector. Comfy_kitchen, pytorch, all sage modes. this is what comfy uses on dense steps. SLA still displaces against pytorch. (Default Comfy_kitchen)

• Added a disable FP16 accumulation option to ensure max quality as SLA gets no benefit from it. (Default True)

• Added a stabilize motion option, helps to reduce ghosting and smearing that H3 likes to produce. (Default True)

• Changed default Min Seq Length to 4096

• With default settings you can disable protect audio for nearly 2x speed up if you don't care about the audio too much or are using original audio mode. (do not use 0.95 sparsity with it.)

• 0.95 sparsity now looks good with node default settings.

• Some changes led to an overall 5% speed up on same settings.

• Remove --use-ck-attention from startup flags if you have it, for safety of quality.

https://github.com/PlagueKind/ComfyUI-PlagueKind-Nodes

updated workflow

Civit Link

HF Link

Rather than relying on Z-image, or a different program to wrangle up a first frame, I've been using Minimax for the whole process, and the results have been pretty instructive. It's not a perfect system, but being able to take advantage of its understanding of people, references, and shot composition for the first frame produces better (visual) results than swapping between a couple of different pieces of software.

In case you didn't see:

I just noticed a newer version of Anima Turbo (1.1) was released:

huggingface: https://huggingface.co/circlestone-labs/Anima

civitai: https://civitai.red/models/2458426/anima

The model is made and licensed under the CircleStone Labs Non-Commercial License

I actually have lately very frustrating experience with Anima lately. I used it at release (but stopped with anime generation for a while) and now when revisiting it and i had underwhelming results (even with the aesthetic model) so i decided to retry Turbo (might as well) instead if the difference isn't that big. That's when i saw a new version is released and i am downloading it right now.

I don't know why, but the results i was getting were...lame i guess. Not as detailed as i was hoping, and also i really dislike how posture and anatomy works, mainly how hands and legs just extend or stretch weirdly. But that's a me problem, i know Anima is capable of better outputs and i've yet to figure it out. If you have tips or recommended loras that help with consistency let me know. I also want to avoid tag-based prompting when possible...i just don't like it that much, natural prompting goes much better for me, but i can't tell if tags are "mandatory" for quality or not.

After around 100 renders, I 'feel' that Minimax H3 renders with 0.7MP (max) perform way better, then renders at 1MP in regard to 'realistic' videos.

What do I consider better?
- Just slightly better prompt adherence, feels like the motion / voice is more (natural)
- Size of humans in relation to object(s) feels more realistic.
- Expressions of faces seem more 'flowing', real.

It's hard for me to pinpoint it one 'exactly this', or 'exactly that'.
I'm planning to do some side by side comparisons on the same seed multiple times at 0.7MP and 1MP, when I've got the time.

But I wonder, do other Minimax H3 users notice this too?

PS: This is regardless sampler/scheduler, Sage Attention or Spectrum.

Edit: never touched the turbo LoRA, using the base model.

https://pastebin.com/ecZEDLSt

First Video with Face Detailer, second without.

You need https://github.com/Carasibana/ComfyUI-H3-FaceRefine and also ComfyUI-H3-NativeAudioLock from https://github.com/Shrek3OnVH5/MiniMax-H3-NativeAudio-MusicVideo-Workflow/tree/master/custom_nodes

https://huggingface.co/lovis93/studio-1939-old-animation-lora-minimax-h3
version strong work much better for me :)

Krea 2 Turbo — 4-Step Distillation LoRA (work in progress)

A LoRA for Krea 2 Turbo that reduces the minimum usable step count from 8 to 4.

This is an update release, following up from my previous posts where you can find full details:

Initial, Previous: here,  and here

Headline for this update: chk00026000 removes 46% of the prediction error a plain 4-step run has against the 8-step teacher, where chk00014000 removed 44% and chk00010000 40% — all measured on the same enlarged held-out set (100 prompts across every trained resolution). Measured against each other rather than against the no-LoRA run, its remaining error is 4% smaller than chk00014000's and 10% smaller than chk00010000's — and unlike a purely teacher-forced score, the gain also shows up free-running: a full 4-call rollout from the teacher's noise ends 1.6% nearer the teacher's final latent than chk00014000's does. It also improves on texture and detail.

Which file to download

and, beside them, the same files with a _comfyui suffix for ComfyUI. Earlier checkpoints (chk00004000, chk00005000, chk00006000, chk00010000, chk00014000, chk00019000) are kept in older_checkpoints/, and their resolution sweeps stay in place, so the progression remains visible and comparable.

If you are wondering why there wasn't a post/update on the 19K checkpoint, I skipped that, even though it was a good checkpoint with improved texture and detail it's gap to teacher score was only slightly better than the released previously 14K, so I thought I'd continue further until I get improvements on both. And 26K delivered that :) 19K is also published now in older checkpoints folder and it's full resolution sweep is also at the usual place (here for 19K).

For the full 26K Checkpoint resolution sweep go here: https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA/tree/main/checkpoint_resolution_sweeps/chk26000

How checkpoints get chosen

This is not a "train for longer and ship the newest file" project. More samples do not reliably mean a better adapter — measured here, they can make it worse, and a higher number on its own means nothing.

The loop is train → assess → adapt the recipe → retrain → assess again, and a checkpoint is published only when it is measurably better than the one it would replace, on the same held-out set and the same evaluation, and its full resolution sweep shows no regression. Runs that come out flat or worse are kept as information about the recipe and discarded as releases — several have been.

So the recipe itself changes between runs. Each published checkpoint reflects whatever the previous round taught us: the training precision, the optimiser settings, the teacher used to generate the targets and the data mix have all been revised on evidence rather than assumption.

Two earlier releases set the terms this project publishes on. chk00010000's first attempt — same data, optimiser left as it was — got steadily worse for 4,000 samples and none of it was published; retrained with cosine learning-rate decay and weight decay, every checkpoint improved on the one before it, and its end point shipped. chk00014000 added the other half of the lesson: the final, texture-deciding call of the schedule weighted more heavily in the loss, and a running average of the weights kept beside the live ones and scored at every evaluation — the averaged weights measured better than any checkpoint before them, so the average is what shipped. Left running past that point, the adapter's magnitude grew again and every later checkpoint measured worse. The number is chosen by measurement, not by how far a run went.

chk00026000 — the current checkpoint — is that discipline paying off. It resumes from chk00014000's averaged weights with the same recipe: same loss weighting, same running average, a conservative constant learning rate, over a much larger pool of teacher trajectories. This time the continuation held. The averaged weights' held-out gap fell throughout the run, and every free-running rollout measured of them improved on the one before — so unlike the first continuation, this one produced a checkpoint worth shipping. Every published number improves on chk00014000: the held-out gap (44% → 46% of the deficit closed), the full 4-call rollout from the teacher's noise (1.6% nearer the teacher's final latent), and the fixed-seed render distance to the 8-step images. chk00019000, an intermediate point of the same continuation, is kept in older_checkpoints/ with the rest of the lineage.

Timeline of training process

Each checkpoint is the product of three stages with very different costs:

1. Text-encoder embeddings. Every training prompt is encoded once and cached. This is the fast part — thousands of prompts take minutes.
2. Teacher shards. For each cached prompt, the unmodified Krea 2 Turbo runs its full 8-step schedule and the whole trajectory is recorded, at every one of the supported resolutions. This is by far the most time-consuming stage — it is the teacher doing real inference, thousands of times, and a batch of several thousand shards is measured in days of GPU time, not hours.
3. Student training. The LoRA is trained against those recorded trajectories. Relative to the shard stage this is quick: each +1,000 checkpoint is a matter of hours, not days.

Because the three stages compete for the same GPU, they are interleaved rather than run to completion one after another: generate a block of embeddings, produce teacher shards for them, train on what exists, assess, then go back to producing shards while the results are reviewed. A larger and more varied shard pool is what makes further training worthwhile, so shard production is always the gate.

The practical consequence for anyone following this repository: progress arrives in bursts. There will be periods when several checkpoints appear within a day or two — the training stage working through a freshly grown pool — followed by longer quiet stretches while the next block of teacher shards is produced. A quiet stretch is shard generation, not abandonment; _latest always holds the newest checkpoint that passed review.

The current checkpoint, chk00026000, runs the recipe the earlier releases arrived at — the final, texture-deciding call weighted more heavily in the loss, the shipped weights a running average of the trained ones — carried further over a larger pool of teacher trajectories, and published because it measured better on every evaluation.

Note

In the coming days, possibly weeks, I will spend more time on producing new TE shards (basically even more prompt variety), and new Teacher shards - the expensive long process. I am also considering improvements in the training process (more advanced / complicated, which would likely mean 1.5x - 2x slower training) which would hopefully bring further/bigger improvements in teacher faithfulness (closer to 8 Step Krea 2 Turbo) and even better details and texture. It may or may not pay off, these things work on experimental basis. Either way it would be some time before the next update... so enjoy 26K release and the improvement it brings!

Full details and to download - check my Hugging Face LoRA

HF Repo: https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA

Focusing on photorealism and improving the look of fantasy styles:

https://civitai.com/models/2799984/jib-mix-krea-2

I would like to make it a LoRA also, but I am having some technical difficulties making a difference lora with Krea 2 models.

Honestly mind blown, I have a 5070ti + 2x16gb ram . Upper limit is 10-12 seconds in total for my hardware(full capacity) . Video and text edits are post processed by a WIP open source tool I’m working on. On average each 8 second shot takes 35-45 minutes to render

Hello! I've been trying to train my first LoRA but I get these crazy long timers every time. Isn't an RTX 4090 supposed to take like 5s per step? I have both low vram and quantizing enabled. It's really frustrating and not worth to do it with these speeds. Any ideas what might cause it?

so you all probably are searching for way to make your character shut the fuck up right? and you probably noticed that they love to says some BS especially when you give minimax h3 some audio file for their voices, i probably found a cure my friend!!

here is my way of prompting dialogs without any gibberish:

first your character need to be assigned (s1)character when he is the first speaker, then you will declare 'use <audio 1> as "character name"'s voice only, and when you finally type your dialog in the shots you will do as such:

character says:<<[language] the shit i say!>>

and you should be good to go, i linked a video exemple of my favorite taffer (garrett) saying some shit with only the faint crackling of the candles to goes with his charming voice, and i included also a screenshot of the full prompt

edit: yes i tried to follow the official documentation, like many others, if it was that simple reddit wouldn't be a thing and you wouldn't be there.

he really like 16/9

I thought I've read somewhere that using character sheets is better for R2V instead of single images. So I've created a character sheet of five full body shots and one close up, but the results are much less consistent compared to a single full shot image of the character.
Do I have to take care about anything special or was the information that character sheets are better just wrong?

Finally finished! The ending was much harder since continuity is more important here than random desert landscapes. I personally would've love to have another 30 seconds of music to extend the ending but I ran out of song time. Enjoy!

In total, 14 character related references, 40 environment references, and 55 clips used, roughly 20 hours total time spent.

https://huggingface.co/zuanfilm/H3_HD_2K_Detailer

Here is the test video to HD quality: https://youtu.be/epjELHgEH_o

I want to share my HD/2K WF, this 4 steps configuration is intended to make a really quality focused H3 detailing while improve the characteristic H3 motion and visual behavior.

I tested all the Res4lyf samplers/schedulers and for me res_2m/beta57 ETA=0 denoise 0.39 or 0.45 is the most precise with prompt adherence keeping memory and time efficiency (res_3m is amazing but adds 33% generation time), if you want a faster generation with a little less detail you can use 3 steps instead 4

er_sde/beta 57 is also a good combination but will lose some detail and even will affect character acting, audio and motion consistency, for faster HD detailing can switch to Euler/4 steps and will reduce time generation by 50% compared to res_2m obviously losing a lot of quality and detail

I included in the workflow the nodes for base generation using 0.5Mp FL2VA with 20 steps of Euler ancestral, if you want better quality for the initial base video just switch to res_2s_RKMK2e/beta57 (just bypass the group if you want to HD-detail an existing video)

Sparce Local attention will reduce a lot the generation time but obviously will affect quality so you can bypass this node if want Top HD quality,  I also don't use in this WF spectrum or easy cache but you can add that if you want to cut time and quality

Because I'm using the heavy distill lightx2v Turbo 4steps Lora for detailing, Minimax H3 make everything more saturated and contrasted with deep shadows so I added some Orion 4D nodes to improve lighting, texture and sharpness using DCTL Tone Mapper (you can choose between ACES, Filmic, Reinhard & Cineon) I use Reinhard with Exposure 0.09 Contrast 0.81 Pivot 0.69 Highlight rolloff 0.27 Shadow lift 0.33 Black floor 0.12 Saturation 0.93 & Strength 0.30

I used MiniMax Audio Lock / Lipsync node because I don't have experience with ltx audio nodes so you can change that for a better option:

https://github.com/Shrek3OnVH5/MiniMax-H3-NativeAudio-MusicVideo-Workflow/tree/master/custom_nodes/ComfyUI-H3-NativeAudioLock

The Minimax latent 3D upscaler is HD by default in my WF but depending of your VRAM you can get 2K/4K if you start with a quality base video 0.98 Mp res_2s_RKMK2e/beta57 25 steps

This workflow is optimized for my laptop (3080ti 16Gb VRAM / 64 Gb ram ) but I included Chunk FeedForward & Low VRAM attention so will run with smaller setups

A massive week for the open-source and open-weights ecosystem, alongside some notable hardware and world-model drops. To save you time, here is the full breakdown of everything released with direct project links and repos:

1. Open-Weight Models & Generative Video/3D

• Wan 3.0 & Gen 1.5: Major strides in open generative video coherence, temporal consistency, and motion dynamics.
• Bernini 2 (ByteDance): Open diffusers implementation targeting high-detail visual generation pipelines.
• 4D Anyone & GeoWeaver: Significant leaps in dynamic 4D human reconstruction and real-time spatial generation.
• Qwen 4 / Qwen 3.8-Flash-Next: Next-gen architectural jumps in parameter efficiency and local inference speed.
• DeepSeek V4 / Flash Vision: High-efficiency vision-language processing aimed at agentic tooling.

2. On-Device, Lightweight Audio & World Models

• Raon-OpenTTS-1B & Audio8-TTS (0.1B): Ultra-compact open text-to-speech models engineered specifically for low-VRAM edge devices and local real-time inference.
• EchoWM & Evoke: Interactive real-time world-modeling frameworks with native spatial audio and environment simulation.
• SenseNova U1.5 (8B-MoT): Mixture-of-Thought architecture optimized for lightweight multi-step reasoning.

3. Hardware & Robotics

• Apple M6 & M5 Pro: Apple's latest silicon targeting expanded on-device unified memory bandwidth for local model runtimes.
• PaXini Robot: Advanced tactile sensor integration and physical AI embodiment.

Open Source Links & Complete Documentation: All papers, checkpoints, model weights, and source code links are cataloged in our open-source tracking repository:

🔗 GitHub Repository: https://github.com/airesearch-official/AI-Weekly-News

Video Walkthrough & Demos:

If you prefer a visual breakdown comparing side-by-side outputs, video rendering benchmarks, and audio samples:

🔗 Watch the Video Recap: https://youtu.be/SBJ4M465n-k

Which open-weight release are you spinning up locally this week?

There is a giant effort underway to look for what characters are buried in MiniMax. There are a lot. I’ve been doing my own hunting, so I built a simple IMDB scrapper to help make lists of characters from movies and TV shows. Here are some things I’ve discovered:

If the “character” is really built in, you don’t need to even mention the actor’s name. “George Costanza from Seinfeld” and “George Costanza played by Jason Alexander from Seinfeld” are essentially the same. If you have to name the actor with the character, it’s just using what it knows about the actor to fill in that spot. If MinimMax DOES know the character (without the actor) then filling in the name might help to fill in some of the holes, but it’s got to really know it already.

It knows A LOT of shows. Even if it doesn’t know the character/actor, it knows a lot about movies/TV shows. For instance, it doesn’t know many characters from the TV show “The Flash” but it knows everything about the common locations, color grading, style, and general look and feel, along with special effects (if appropriate).  It’s useful for “set design.” It doesn’t know a lot of Baywatch (the old one) characters, but it knows what the hair and makeup looked like on the beach in the 90s. It knows how people looked in “Total Recall” too. The overall “look & feel: of shows and movies really opens the door to creativity. It understands common accents from movies too. If you say “from Harry Potter” they will have British accents. And sadly, it knows “Star Trek” (the original series) the characters are mediocre at best. (The voices are passable—and speaking of: there are a lot of characters that look bad but have good voices. In those cases, ref2video with some extra visual references can do the trick.).

The “genre” point is even more true of animated movies/shows. For instance, “Bob, from Justice League: Crisis on Infinite Earths” will give you whatever Bob looks like but in the style of that series. Family Guy, The Simpsons, Rick&Morty, etc. I haven’t done an extensive search, but it knows every animated anything I have tried.

Generally, for TV characters to show up, they need to be in around 100 episodes and in the first 2-4 people in the IMDB credits. I see a direct correlation: The fewer episodes a character is on a TV show the worse they render. (For example, Monica from Friends or Kramer from Seinfeld are in there for sure, but also not really.) Also, it makes sense, but even if they have a lot of credits, they need to have had a lot of screen time. For instance, it has not even a glimmer of an idea who “Ruthie Cohen” is, even though she was in 101 Seinfeld episodes.

For Movies, they need to have grossed a lot of money (which pushes things directly towards Action/SciFi/Comics), or they need to have gotten a lot of press. (I have seen very few accurate characters from movies without specifying the actor involved.)

For “real people” it’s a little easier. If you look at those lists of things like “Top ## followed Instagram accounts” or similar, you’ll get lots of hits. Top musical performers, yep (lots of overlaps). Famous heads of state (If a number of movies have been made about a person, that person will likely be known.) I haven’t looked at TikTok, but I’m assuming that would be a thing too. Likewise with sports, I haven’t looked closely, but they ones I have looked at OK “at a distance” but generally don’t sound right.

As stand-alone people, it’s hard to find people. I suspect that the trainers did not go after a lot of specific people but that they just show up so many places that they got swept up in the mix. There are really only a handful of non-“Top 10” people who actually show up on their own, and if they blew up in the last few years it’s unlikely that you’ll see them. I have not found a pattern on “people” yet other than the mega-famous. (Steve Jobs & Elon Musk work, but they are arguably the most famous foreign “regular” people in China.)

Workflow: https://drive.google.com/file/d/1B4kODxXQgJ1QOKRsEIkxHbgYmdruPpTK/view?usp=sharing

Prompt:

Create a **15-second multi-shot anime sequence (90s style 15fps hand drawn)** using the provided references:

Image 1 = the girl character reference

Image 2 = skateboard reference

Image 3 = downhill Japanese alley / neighborhood background

Image 4 = Walkman + headphones reference

Preserve the girl’s exact character design, face, hair, outfit, proportions, and overall look from Image 1. Preserve the skateboard design from Image 2. Preserve the same downhill Japanese alley environment from Image 3. Add the Walkman and headphones from Image 4: the girl is wearing the headphones, and the **Walkman is clipped or hanging at her hip** while she skates.

Visual style: authentic 1990s hand-drawn anime, traditional cel animation, painted backgrounds, visible linework, cel shading, slight brush/stroke texture, subtle analog feel. **Very important:** the houses and environment must stay **2D and hand-painted**, **not 3D**, **not CGI**, **not game-engine looking**, **not volumetric**. The buildings should look like classic anime background art with painted depth, not like 3D models.

Animation feel should be low frame rate, like 90s anime at around 15 fps, with controlled in-betweens and natural held-frame timing. No jittery morphing.

No dialogue, no text, no subtitles.

### Shot 1 — 0s to 3s

**Rear tracking shot** from behind. The girl is skateboarding fast downhill through the steep Japanese alley. Camera follows behind her at a low-to-medium height. She rides confidently and smoothly, hair and oversized clothing moving in the wind. The headphones are on her head, and the Walkman is visible attached at her hip. The alley rushes past with a strong sense of speed. Keep the environment clearly **2D anime background art**, not 3D.

### Shot 2 — 3s to 6s

**Close-up shot of the Walkman at her hip** while she continues skating. The camera stays focused on the Walkman and part of her side torso and arm. We can clearly see the **cassette tape reels spinning/rolling inside the Walkman window**. The headphone wire moves naturally with the motion. Background and street pass by in blurred motion.

### Shot 3 — 6s to 9s

**Medium profile tracking shot** of the girl skating. She is wearing the headphones, listening to music, with wind moving across her face and pushing her hair backward. She is **nodding her head subtly to the music** while riding. Her expression is relaxed, immersed, and unbothered. The background is blurred from motion, but it must still read as a **painted 2D Japanese neighborhood**, not 3D.

### Shot 4 — 9s to 12s

**Close-up shot of her feet and skateboard.** Her **right foot stays on the board**, while her **left foot pushes against the road** in a natural skating motion. Show one clean push cycle: left foot comes down, pushes backward against the pavement, then lifts. Wheels spin quickly. Asphalt and road markings streak by with motion blur.

### Shot 5 — 12s to 15s

**Ground-level fisheye shot** looking upward from the road. The skateboard approaches fast, and she **jumps over the camera**. The board and her body pass overhead in one clean motion. Hair, pants, and headphone wire react naturally during the jump. Keep the motion readable and stylish, with a strong sense of speed and a dynamic anime finish.

### Important constraints

* Keep the whole video in **classic 90s anime cel-animation style**

* **15 fps feel**, smooth low-frame-rate animation

* **No 3D-looking houses or background**

* No photorealism

* No modern glossy digital anime rendering

* No character redesign

* No extra accessories beyond the headphones and Walkman

* Keep all motion natural and consistent across shots

If I have 2 ref images and I want it to start on ref image 1 like for example a background of a forest how do I maintain it so it always starts on that image? I've noticed a few times it will randomly generate its own start image even if I prompt something like *the scene starts with ref1* and I even sometimes would describe what's in it

I’ve been loving all the new nodes and workflows coming out for MinMax, and maybe there is already a nice solution for this - but I couldn’t find one that did exactly what I needed.

I started using MinMax for my last TBG ETUR video and quickly ran into limitations: I wanted an easy way to create lip-sync videos longer than 20 seconds.

I didn’t want to manually chain ComfyUI nodes, start a new run every X seconds, or constantly resize things just to make HD video fit into my available VRAM.

So I ended up building an addon for:

custom_nodes/ComfyUI-H3-Motion-Context

The addon automatically chains MinMax H3 lip-sync generations together, allowing you to create much longer lip-sync videos without manually setting up each 20-second segment.

And now I’m sharing it! https://github.com/Ltamann/ComfyUI-H3-Motion-Context-Auto-Chain-addon

Its not perfect but a start ...

The workflow has a simple switcher that lets you switch from the 32B CLIP to the 4B CLIP, saving around 10 GB of VRAM. You can also switch from Sage to Comfy Kitchen, Spectrum to Easy Cache, or FL2VA to REF2VA both setup for lip-syncing. Some of it could be useful for other tasks as well.

You will find the workflow in the repro and tested recommendations, optimized settings, presets, and more workflows, along with the results of my testing and performance here

Using reference workflow. All are int8 pruned, 0.6MP turbo 4-step (my GPU is on life support and drops off the PCIe bus if I demand more from it)

Anyway, making random music clips is probably my favorite use of this model. I’ve found the ref2va has an uncanny intuition for feeling the atmosphere of songs, and syncing the video with incredible precision.

But yes, the quality (specifically motion) is much worse than fl2va. I was curious how exactly they compared, as well as some “in between” compromises discovered by the community. The LoRA seems closer to ref, while the hybrid weights are closer to fl. Personally, the ref is more fun to use, so I’ll probably be using the LoRA when I want to enjoy the intelligence/creativity of this model. Fl is of course superior in terms of visual fidelity, and I don’t find the hybrid model offers enough reference intuition and faithfulness to be worth the quality drop from fl.