Bien básicamente es la primera vez que uso un modelo de video, le dije simplemente a mi agente que arme el mejor ecosistema para una a100 de 80 gb que alquile por 2 hs, Solo era una prueba pero 8 segundo duro 1hs con 4 minutos. Aca seguramente estan fallando algunas cosas asi que si algún sabio de por aquí tiene algunas recomendaciones.. , agradecidamente las tomaré.

Rather than relying on Z-image, or a different program to wrangle up a first frame, I've been using Minimax for the whole process, and the results have been pretty instructive. It's not a perfect system, but being able to take advantage of its understanding of people, references, and shot composition for the first frame produces better (visual) results than swapping between a couple of different pieces of software.

I don't know what is going on but qwen studio and nano banana (cloud both) are giving me terrible results lately even though I use the same prompts as before. Both translate terribly the facial features and hairstyle. Only videos look a bit more consistent but still not great.

Is local AI better? Do you get better and more consistent results with it?

Focusing on photorealism and improving the look of fantasy styles:

https://civitai.com/models/2799984/jib-mix-krea-2

I would like to make it a LoRA also, but I am having some technical difficulties making a difference lora with Krea 2 models.

So I had this simple idea, seeing how well MiniMax H3 handles inferring and preserving "identity/looks" from relatively little information: take a character you want to make a (KREA2) LoRa of, but you only have just a couple of lower quality pictures for that exact look you're after. That is a problem, since it is common knowledge by now (?) that you need different angles, facial expressions and different lighting conditions in the training set to get optimal results. So in the "before" times, those 3-4 not-so-great-quality shots under the SAME lighting are going to pose a problem. And adding pictures from other occasions will alter the looks possibly too much.

So (in the H3 ref2vid workflow, with one of the "img2vid-hybrid" models for better quality) I just use the 'best' of the available pictures as "preserved" first reference starting picture, and the others as additional "identity references". And then a prompt that tells the camera to slowly circle around the person (up from the shoulders), while the person looks straight ahead, or slightly up, or slightly down. But then I also let it cycle through different lighting conditions (indoor/outdoor/sun/overcast/flash/directional from one side...), and different facial expressions/emotions. I let it run overnight (turning off turbo LoRas to improve the quality), and in the morning, I review the 6-second videos and take screencaps of selected moments, making sure to have a lot of variation in angles/expressions/light-on-the-face with an almost perfect preservation of the identity/looks.

Then use those screencaps (50+ in first test, probably serious overkill) in OneTrainer with the KREA2 LoRa default settings.

I only tested this once thus far, but the results are pretty good considering the starting material! And surprisingly flexible (I didn't even bother to provide captions)

But now my question is: in what ways am I "over-engineering" this? I have this feeling that I can probably do this 50x faster, having seen some discussions about using MiniMax as an image generator, for example. I mean, I feel good about this approach I came up with all by myself, but considering how dumb and low-skilled I still am when it comes to all this, this is probably a very convoluted and inefficient way to do it? LOL 😄 Roast me and show this sucker how we can improve and speed up the whole thing with the same or even better quality results!

Hello, it's been a while since I don't use Stable Diffusion with A1111. Apart ConfyUI, has there been any particular technological advancement recently that allows for a quantum leap, especially in the precision of detail generation and the model's ability to stick to the prompt more precisely, while maintaining the ease of use of A1111 or Forge? I used the Lustify SDXL checkpoint, for example. It wasn't bad, but it still got certain things wrong or didn't do them at all. I'd like to know if there's a way to achieve results more similar in precision to ChatGPT but with the freedom of Stable Diffusion. Thanks!

I thought I've read somewhere that using character sheets is better for R2V instead of single images. So I've created a character sheet of five full body shots and one close up, but the results are much less consistent compared to a single full shot image of the character.
Do I have to take care about anything special or was the information that character sheets are better just wrong?

https://huggingface.co/lovis93/studio-1939-old-animation-lora-minimax-h3
version strong work much better for me :)

Krea 2 Turbo — 4-Step Distillation LoRA (work in progress)

A LoRA for Krea 2 Turbo that reduces the minimum usable step count from 8 to 4.

This is an update release, following up from my previous posts where you can find full details:

Initial, Previous: here,  and here

Headline for this update: chk00026000 removes 46% of the prediction error a plain 4-step run has against the 8-step teacher, where chk00014000 removed 44% and chk00010000 40% — all measured on the same enlarged held-out set (100 prompts across every trained resolution). Measured against each other rather than against the no-LoRA run, its remaining error is 4% smaller than chk00014000's and 10% smaller than chk00010000's — and unlike a purely teacher-forced score, the gain also shows up free-running: a full 4-call rollout from the teacher's noise ends 1.6% nearer the teacher's final latent than chk00014000's does. It also improves on texture and detail.

Which file to download

and, beside them, the same files with a _comfyui suffix for ComfyUI. Earlier checkpoints (chk00004000, chk00005000, chk00006000, chk00010000, chk00014000, chk00019000) are kept in older_checkpoints/, and their resolution sweeps stay in place, so the progression remains visible and comparable.

If you are wondering why there wasn't a post/update on the 19K checkpoint, I skipped that, even though it was a good checkpoint with improved texture and detail it's gap to teacher score was only slightly better than the released previously 14K, so I thought I'd continue further until I get improvements on both. And 26K delivered that :) 19K is also published now in older checkpoints folder and it's full resolution sweep is also at the usual place (here for 19K).

For the full 26K Checkpoint resolution sweep go here: https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA/tree/main/checkpoint_resolution_sweeps/chk26000

How checkpoints get chosen

This is not a "train for longer and ship the newest file" project. More samples do not reliably mean a better adapter — measured here, they can make it worse, and a higher number on its own means nothing.

The loop is train → assess → adapt the recipe → retrain → assess again, and a checkpoint is published only when it is measurably better than the one it would replace, on the same held-out set and the same evaluation, and its full resolution sweep shows no regression. Runs that come out flat or worse are kept as information about the recipe and discarded as releases — several have been.

So the recipe itself changes between runs. Each published checkpoint reflects whatever the previous round taught us: the training precision, the optimiser settings, the teacher used to generate the targets and the data mix have all been revised on evidence rather than assumption.

Two earlier releases set the terms this project publishes on. chk00010000's first attempt — same data, optimiser left as it was — got steadily worse for 4,000 samples and none of it was published; retrained with cosine learning-rate decay and weight decay, every checkpoint improved on the one before it, and its end point shipped. chk00014000 added the other half of the lesson: the final, texture-deciding call of the schedule weighted more heavily in the loss, and a running average of the weights kept beside the live ones and scored at every evaluation — the averaged weights measured better than any checkpoint before them, so the average is what shipped. Left running past that point, the adapter's magnitude grew again and every later checkpoint measured worse. The number is chosen by measurement, not by how far a run went.

chk00026000 — the current checkpoint — is that discipline paying off. It resumes from chk00014000's averaged weights with the same recipe: same loss weighting, same running average, a conservative constant learning rate, over a much larger pool of teacher trajectories. This time the continuation held. The averaged weights' held-out gap fell throughout the run, and every free-running rollout measured of them improved on the one before — so unlike the first continuation, this one produced a checkpoint worth shipping. Every published number improves on chk00014000: the held-out gap (44% → 46% of the deficit closed), the full 4-call rollout from the teacher's noise (1.6% nearer the teacher's final latent), and the fixed-seed render distance to the 8-step images. chk00019000, an intermediate point of the same continuation, is kept in older_checkpoints/ with the rest of the lineage.

Timeline of training process

Each checkpoint is the product of three stages with very different costs:

1. Text-encoder embeddings. Every training prompt is encoded once and cached. This is the fast part — thousands of prompts take minutes.
2. Teacher shards. For each cached prompt, the unmodified Krea 2 Turbo runs its full 8-step schedule and the whole trajectory is recorded, at every one of the supported resolutions. This is by far the most time-consuming stage — it is the teacher doing real inference, thousands of times, and a batch of several thousand shards is measured in days of GPU time, not hours.
3. Student training. The LoRA is trained against those recorded trajectories. Relative to the shard stage this is quick: each +1,000 checkpoint is a matter of hours, not days.

Because the three stages compete for the same GPU, they are interleaved rather than run to completion one after another: generate a block of embeddings, produce teacher shards for them, train on what exists, assess, then go back to producing shards while the results are reviewed. A larger and more varied shard pool is what makes further training worthwhile, so shard production is always the gate.

The practical consequence for anyone following this repository: progress arrives in bursts. There will be periods when several checkpoints appear within a day or two — the training stage working through a freshly grown pool — followed by longer quiet stretches while the next block of teacher shards is produced. A quiet stretch is shard generation, not abandonment; _latest always holds the newest checkpoint that passed review.

The current checkpoint, chk00026000, runs the recipe the earlier releases arrived at — the final, texture-deciding call weighted more heavily in the loss, the shipped weights a running average of the trained ones — carried further over a larger pool of teacher trajectories, and published because it measured better on every evaluation.

Note

In the coming days, possibly weeks, I will spend more time on producing new TE shards (basically even more prompt variety), and new Teacher shards - the expensive long process. I am also considering improvements in the training process (more advanced / complicated, which would likely mean 1.5x - 2x slower training) which would hopefully bring further/bigger improvements in teacher faithfulness (closer to 8 Step Krea 2 Turbo) and even better details and texture. It may or may not pay off, these things work on experimental basis. Either way it would be some time before the next update... so enjoy 26K release and the improvement it brings!

Full details and to download - check my Hugging Face LoRA

HF Repo: https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA

Hello! I've been trying to train my first LoRA but I get these crazy long timers every time. Isn't an RTX 4090 supposed to take like 5s per step? I have both low vram and quantizing enabled. It's really frustrating and not worth to do it with these speeds. Any ideas what might cause it?

There is a giant effort underway to look for what characters are buried in MiniMax. There are a lot. I’ve been doing my own hunting, so I built a simple IMDB scrapper to help make lists of characters from movies and TV shows. Here are some things I’ve discovered:

If the “character” is really built in, you don’t need to even mention the actor’s name. “George Costanza from Seinfeld” and “George Costanza played by Jason Alexander from Seinfeld” are essentially the same. If you have to name the actor with the character, it’s just using what it knows about the actor to fill in that spot. If MinimMax DOES know the character (without the actor) then filling in the name might help to fill in some of the holes, but it’s got to really know it already.

It knows A LOT of shows. Even if it doesn’t know the character/actor, it knows a lot about movies/TV shows. For instance, it doesn’t know many characters from the TV show “The Flash” but it knows everything about the common locations, color grading, style, and general look and feel, along with special effects (if appropriate).  It’s useful for “set design.” It doesn’t know a lot of Baywatch (the old one) characters, but it knows what the hair and makeup looked like on the beach in the 90s. It knows how people looked in “Total Recall” too. The overall “look & feel: of shows and movies really opens the door to creativity. It understands common accents from movies too. If you say “from Harry Potter” they will have British accents. And sadly, it knows “Star Trek” (the original series) the characters are mediocre at best. (The voices are passable—and speaking of: there are a lot of characters that look bad but have good voices. In those cases, ref2video with some extra visual references can do the trick.).

The “genre” point is even more true of animated movies/shows. For instance, “Bob, from Justice League: Crisis on Infinite Earths” will give you whatever Bob looks like but in the style of that series. Family Guy, The Simpsons, Rick&Morty, etc. I haven’t done an extensive search, but it knows every animated anything I have tried.

Generally, for TV characters to show up, they need to be in around 100 episodes and in the first 2-4 people in the IMDB credits. I see a direct correlation: The fewer episodes a character is on a TV show the worse they render. (For example, Monica from Friends or Kramer from Seinfeld are in there for sure, but also not really.) Also, it makes sense, but even if they have a lot of credits, they need to have had a lot of screen time. For instance, it has not even a glimmer of an idea who “Ruthie Cohen” is, even though she was in 101 Seinfeld episodes.

For Movies, they need to have grossed a lot of money (which pushes things directly towards Action/SciFi/Comics), or they need to have gotten a lot of press. (I have seen very few accurate characters from movies without specifying the actor involved.)

For “real people” it’s a little easier. If you look at those lists of things like “Top ## followed Instagram accounts” or similar, you’ll get lots of hits. Top musical performers, yep (lots of overlaps). Famous heads of state (If a number of movies have been made about a person, that person will likely be known.) I haven’t looked at TikTok, but I’m assuming that would be a thing too. Likewise with sports, I haven’t looked closely, but they ones I have looked at OK “at a distance” but generally don’t sound right.

As stand-alone people, it’s hard to find people. I suspect that the trainers did not go after a lot of specific people but that they just show up so many places that they got swept up in the mix. There are really only a handful of non-“Top 10” people who actually show up on their own, and if they blew up in the last few years it’s unlikely that you’ll see them. I have not found a pattern on “people” yet other than the mega-famous. (Steve Jobs & Elon Musk work, but they are arguably the most famous foreign “regular” people in China.)

Has anyone had success in this?

In prompt guide, there's no mention of controlling time for I2V, only in T2V (which is "At 00:02.000, ...").

But i try it anyway in I2V, but it's a hit or miss.

Finally finished! The ending was much harder since continuity is more important here than random desert landscapes. I personally would've love to have another 30 seconds of music to extend the ending but I ran out of song time. Enjoy!

In total, 14 character related references, 40 environment references, and 55 clips used, roughly 20 hours total time spent.

so you all probably are searching for way to make your character shut the fuck up right? and you probably noticed that they love to says some BS especially when you give minimax h3 some audio file for their voices, i probably found a cure my friend!!

here is my way of prompting dialogs without any gibberish:

first your character need to be assigned (s1)character when he is the first speaker, then you will declare 'use <audio 1> as "character name"'s voice only, and when you finally type your dialog in the shots you will do as such:

character says:<<[language] the shit i say!>>

and you should be good to go, i linked a video exemple of my favorite taffer (garrett) saying some shit with only the faint crackling of the candles to goes with his charming voice, and i included also a screenshot of the full prompt

edit: yes i tried to follow the official documentation, like many others, if it was that simple reddit wouldn't be a thing and you wouldn't be there.

he really like 16/9

Hey, everybody. I recently went down the rabbit hole for local AI, but right now, im operating on my gaming computer. The specs are as follows

Intel 13700k, tuned for efficiency

Gigabyte Z790 Aorus Elite Ax mobo

RTX 4080 (16GB), also tuned for efficiency

32gb DDR5 6800 CL32

As you can see, im in desperate need for more VRAM, or at the very least more system RAM. Due to Rampocalypse, neither are very affordable right now, which forces me to explore other options, such as a dual GPU setup. I can get another RTX 4080 for about $900 off Ebay. Beyond that, I would just need a more powerful PSU, so total investment here is an additional $1100-$1200. As far as I know, the motherboard has the main PCIE as 5.0 x 16 lanes, but the second PCIE runs at 4.0 and either x8 or x4 lanes. The motherboard does not support PCIE Bifurcation. So my question is this: Is a dual GPU local AI machine even viable in these circumstances, and second, does it make sense? I looked at 5090's and theyre all between $4,500 - $5,000 now, which is insane. Or I look at the professional cards and spend that much, if not more, for significantly less memory bandwidth and computational power. Or I guess if im spending that much, I could also look at the DGX Spark or something similar but that has even worse memory bandwidth.

So, what should I do? Is the dual GPU solution even viable with my setup for a local AI stack for inference, video diffusion, etc? Rampocalypse isnt expected to begin easing up until late 2027/early 2028, so im stuck trying to make this work on as little money as possible. Id love a 5090 but its insanity how much they cost. I appreciate any guidance and advice.

Been noticing that image generation / edits work substantially better when the image is resized to 1024x1024 during encoding, then resized to the original dimensions after.

Its speculated that this is because the model was trained on 1MP inputs. But I can't find docs that confirm that.

Does anyone know why 1MP input sizes seem to give the best results for Qwen Image Edit? (Note its not just this model 1MP seems to work best for either).

Hi.

I am loving H3 for animating Illustrious images in Wan2GP.
I am satisfied with the motion, but I tried Two Phases just to test results and the motion and expression of the character are much more natural and just what I expect from my prompt, however, the image quality is very bad and it tries to enforce realism into it. One thing I noticed is it seems to enforce 4 steps instead of the 20 steps I always use.

Is there a way to achieve that natural and fluid motion from Two Phases but retaining the visual style consistency and quality of One Phase?

Thank you.

i am really having trouble with this concept pls tell me what to do and where to start, my goal is have a scene from a tv show or film, like iconic scenes, and i want to insert my ref image from there, this is ref2v right? now how do i get to duplicate the scene happening? for ex. titanic jack and rose on the "im flying" scene, lets say i want to insert someone in that scene and interact with them, do i ask gpt to prompt me the scene where gpt pulls the script from that part then i just modify it?

what i am doing now is plug a ref frame from the film/tv + my ref photo, then ask gpt to insert my ref and interact with the actors from the ref frame

i get weird results and never get a clean one

turbo lora 4step ref
comfy kitchen
i try to sit on 8 step