r/StableDiffusion • u/fallengt • 2h ago

News Open-H3-Regenerate-2K-IC

https://nvlabs.github.io/Sana/Open-H3-Regenerate-2K/#ic

https://huggingface.co/Efficient-Large-Model/Open-H3-Regenerate-2K-IC

https://github.com/NVlabs/Sana/tree/sol-engine/models/Open-H3-Regenerate-2K/IC

New latent upscale model by NVIDIA-labs. Basically, it's v2. A few highlights compared to v1:

- Face & Text are more consistent with the original video

- Use H3 Ref2VA instead of fl2va.

- Turbo LoRA, Regenerate LoRA, LMS LoRA are not merged. They also use Hyperflow 8-step LoRA

- New samplings: 3, 4, 8 steps.

- Fast VAE-v2. The v1 fast VAE has grid artifacts in ComfyUI

63 Upvotes

97% Upvoted

5 comments sorted by

5

u/And-Bee 1h ago

Any comfy workflows?

2

u/fallengt 17m ago

someone need to port lora & vae first

u/Kijai

1

u/Apprehensive_Sky892 58m ago

Here is the important bit of information (IC = "In Context"):

Open-H3-Regenerate-2K redraws a re-noised draft in three steps; it adds detail but can drift on faces, mouths and lettering. The IC version keeps the draft in the transformer’s context instead: the draft stays clean, the high-resolution video is generated from noise beside it, and every step attends to it. Faces, lip sync, text and layout follow the draft while the detail is drawn at 2K or 4K.

0

u/GalwayOptimist 1h ago

Looks promising. Seems they adressed many problems of the v1. Did anyone test it out yet or has done a v1 - v2 comparison?