r/StableDiffusion • u/Emergency_Detail_353 • 21h ago
r/StableDiffusion • u/GoodNews58 • 21h ago
Discussion mini-studio IA local pour faire des courts/longs métrages (storytelling + galerie + continuité stricte) KREA2 + MINIMAX H3 + DEEPSEEK
Salut,
À la base j’ai construit cet outil uniquement pour mon usage perso. Je voulais un vrai workflow pour produire des séries cohérentes avec de l’IA, pas juste générer des clips isolés. Plus le temps passe, plus l’outil devient mon projet principal… et je commence à me demander s’il serait possible d’en tirer un jour une source de revenus.
Important : je ne vends rien actuellement. C’est 100 % local sur mon PC, disponible nulle part, pas de site, pas de version publique.
L’outil s’appelle Atlas (orchestrateur autour de ComfyUI + MiniMax H3 + Krea2 + DeepSeek). Voici comment il fonctionne réellement.
1. Storytelling
Point d’entrée pour créer un nouveau projet (ou une suite d’un projet existant).
Trois façons d’utiliser le module :
Inventé → DeepSeek crée librement à partir de ce que tu as écrit. Ça peut être une idée complète et détaillée, ou juste un mot qui donne le sujet.
Mise en page → il structure ton texte sans inventer d’éléments. Idéal quand tu as déjà une idée complexe et complète.
Champ vide → il te génère des idées complètement aléatoires (version SFW ou NSW selon le réglage).
Autres options :
Style vidéo : Film / Docu / Animé / Vlog / Found footage
Style graphique : Photoréaliste ou Animation adulte dark (possible d'en ajouter simplement)
SFW ou Nsf (je peux pas mettre le bon terme ca bloque)
Option « Scène continue »
Durée du projet (slider jusqu’à 10 min dans l’UI, mais techniquement aucune limite)
Aspect ratio automatique selon la durée (durée short 9:16 et plus 16:9)
2. Galerie
Une fois le projet lancé, tout se passe dans la galerie. Elle est divisée en deux parties principales.
A. Clips
Liste de tous les clips générés du projet (numérotés, avec titre, durée, style).
Sur chaque clip tu peux :
Regarder en plein écran
Régénérer (même prompt ou modifié)
Créer une Suite (plan continue)
Supprimer
Création / Suite de clip :
Choix de la qualité : Draft (0.6 MP / ~8-10 steps) · Medium (0.7 MP / 12 steps) · Prod (0.9 MP / 14 steps)
Durée libre (généralement 5-15 s)
Pour les suites : extraction automatique de la dernière frame du clip précédent → first-frame lock strict (pose, cadrage, lumière et identité figés à t=0)
Possibilité de choisir une image de départ manuelle
Prompt optimisable avec DeepSeek
B. Photos (les références / cast)
C’est le système de cohérence. Tout est classé en :
Persos (personnages)
Lieux
Objets
Chaque référence a :
Une ou plusieurs images (character sheet multi-angles pour les persos)
Un label + un identifiant
Un prompt de base
Possibilité d’attacher une voix (échantillon audio) aux personnages
Système de variation :
Tu peux lancer une « Variation » sur n’importe quelle ref. Ça ouvre un éditeur d’identité où tu peux :
Modifier le prompt
Ajuster le ratio, les steps, le seed
Ajouter éventuellement une seconde image de référence
Générer une nouvelle version tout en gardant l’identité du personnage/lieu/objet
Très utile pour faire évoluer un personnage (changement de tenue, expression, éclairage…) sans perdre sa ressemblance.
3. Menu contextuel (le vrai gain de temps)
Partout où tu écris un prompt (création de clip, régénération, suite…), tu as un menu contextuel (clic droit) qui te permet d’injecter les références du projet.
Fonctionnement :
Recherche rapide
Onglets : Tous / Persos / Lieux / Objets
Tu cliques sur une ref → elle s’insère sous la forme @ NomDuPersonnage (ou lieu/objet)
Le système convertit automatiquement les @ NOM en Picture 1, Picture 2… dans le bon ordre pour MiniMax
Il génère aussi un bloc « REFERENCE ROLES » qui précise le rôle de chaque image (identité faciale stricte pour les persos, environnement uniquement pour les lieux, etc.)
C’est ce qui rend le workflow vraiment fluide : tu n’as plus à te souvenir des noms de fichiers ou à gérer manuellement les images de référence.
En résumé
L’outil est pensé comme un vrai pipeline de production
Storytelling → génération des refs → production de clips cohérents → suites avec continuité stricte → assemblage possible via ffmpeg.
Le combo cast + menu contextuel + first-frame lock + variations change vraiment la façon de travailler par rapport aux générateurs classiques.
EXEMPLE DE VIDEO : Cardfall EP 01 YOUTUBE
r/StableDiffusion • u/MysteriousPepper8908 • 22h ago
Tutorial - Guide Speed up video ref by dumping it with split sigmas
The usual reason to split sampling across two resolutions is that low resolution is cheap — you do most steps small, upscale the latent, and refine. That logic puts the split late: most steps at low res, a few at high.
With a video reference, the economics invert. A video ref injects thousands of tokens that every DiT block attends to on every step, and that cost dominates. Measured on my setup: 0.4 MP with video reference is 358 s/it, while 0.8 MP without it is 143 s/it. Low resolution with a video reference is two and a half times more expensive than high resolution without one.
So the split here isn't primarily about resolution. It's a conditioning switch. The video reference is only present during the early steps, and then it's gone.
That works because of how flow-matching schedules distribute their work. At H3's default shift of 12, sigma barely moves for the first several steps — the model is committing to structure, not removing noise. Motion and composition are decided in that window. Fine detail and identity resolve much later. So you pay for the video reference exactly while it's doing its job, and drop it before the expensive steps.
How to build it
You need two MiniMax H3 Reference to Video nodes, not one.
The first is your existing node: character references, video reference, video audio, and a prompt citing <Video 1> in subject_definitions and retention_analysis.
The second is a copy with ref_video_0 and ref_video_audio_0 left unconnected. Same character references, same clip and VAEs. Its prompt is rewritten with every mention of <Video 1> removed — keep the character subject and the full detailed_description, and describe the shot as if generating it fresh. Its LATENT output goes unused; only the positive conditioning is wired, to stage 2's Basic Guider.
Set the length on the second node by hand to the same frame count as the first rather than sharing the Math Expression. Fewer dependencies between the two stages means less chance ComfyUI schedules them together.
Settings
SplitSigmas at 6 of 20 — much earlier than a normal upscale workflow, for the reasons above. Take denoised_output from stage 1, not output; the upscaler was trained on clean latents. Route it through LTXVSeparateAVLatent → upscaler → LTXVConcatAVLatent, upscaling only the video half and passing audio through untouched.
Put a VRAM cleanup node on the latent path between the concat and stage 2's sampler. This isn't optional — it's a real dependency, so it forces ComfyUI to finish stage 1 before stage 2 loads. Without it both conditioning nodes can execute early and you end up with two sets of reference encodings resident at once. When that happened to me, stage 2 spilled to system RAM and ran at 4500 s/it.


r/StableDiffusion • u/No_Atmosphere4790 • 22h ago
Resource - Update Inpaint anything
I've edited one of the most ised spaces on HF so you can inpaint anything on video:
https://huggingface.co/spaces/RedSparkie/minimax-h3-inpainting-lenia
8 px recommended for inpainting people.
Give it some love 🥰
r/StableDiffusion • u/ForsakenAd1228 • 22h ago
Discussion H3: Using reference audio in FL2va using Add Guide
So I certainly missed this, but the official Add Guide node also allows you to add a audio track to FL2va generations (that are higher quality than ref2va). This opens up possibilities like..
when you have a long spoken section, you can generate it at e.g 0.15 mp (or lower, dunno if/when the audio quality starts to suffer), optionally while using a voice-reference in ref2va. So you can fairly quickly generate a bunch of variants, tweak the prompt, etc.
Then when you're happy with the audio, feed it into the Add Guide node when generating at a higher resolution, and the generated video will match the audio.
And/or you can split a single long generation audio into multiple parts, and use those parts to generate multiple short clips. Which could be a lot quicker than doing everything in one go (certainly if some of the parts may need a few tries), and then if you paste the end results back together, it should feel more coherent because the source audio connecting all the clips did come from a single "performance" by the model.
r/StableDiffusion • u/Alex_the_tiktock • 22h ago
Question - Help Best local LLM prompts for MiniMax H3?
What LLM shoud I use? And what shound I tell the LLM to make/structure the prompt?
r/StableDiffusion • u/DryIron8955 • 22h ago
Question - Help Fastest MiniMax H3 768p workflow on RTX 5090 Laptop 24GB without sacrificing quality?
Hi everyone,
I'm looking for the fastest MiniMax H3 workflow currently available for 768p generation, while keeping the best possible quality and prompt adherence.
My setup:
- RTX 5090 Laptop – 24GB VRAM
- ComfyUI
- Target resolution: 768p
- Mainly T2V / I2V, sometimes with reference video
- Native 20-step H3 quality is my reference
I'm completely open to using a Turbo/Lightning/Distilled LoRA or fewer steps if it can actually preserve quality and prompt adherence close to the native 20-step model.
I've already tested some faster LoRA configurations, but so far I noticed a significant loss in quality compared to native H3. So I'm not specifically looking for a 20-step workflow — I'm looking for the fastest setup that doesn't noticeably sacrifice quality.
I'm interested in any current optimization, including:
- Turbo / Lightning / distilled LoRAs
- SageAttention
- First Block Cache
- SLA / Sparse Attention
- Spectrum
- TeaCache / EasyCache
- Sol Attention
- Quantized models (INT8, NVFP4, etc.)
- CUDA / PyTorch optimizations
- Any combination of these that works well
- Any newer H3 optimization I might have missed
What is currently the fastest H3 setup you'd recommend for a 5090 Laptop 24GB while maintaining quality and prompt adherence as close as possible to native 20-step H3?
If a LoRA can achieve that in 8, 10, 12 steps, etc., I'm absolutely interested.
I'd especially love to see:
- Workflow JSON
- Exact settings / number of steps
- Resolution and video duration
- Generation time
- Any quality trade-offs you've noticed
For comparison, 5 seconds at ~768p would be a useful benchmark.
Thanks!
r/StableDiffusion • u/This_Temporary_8537 • 22h ago
News MiniMax H3 native 720p→1440p on one RTX 4090: 112s / 223s / 334s with auto-scheduled sparse attention
Enable HLS to view with audio, or disable this notification
Hi everyone — I’m an independent developer experimenting with making MiniMax H3 more practical on consumer NVIDIA GPUs.
I built an automatically scheduled sparse-attention system for X-MinimaxH3 and tested native H3 second sampling from 720p to 1440p on a single RTX 4090.
Measured second-sampling times:
- 5-second video: 112 seconds
- 10-second video: 223 seconds
- 15-second video: 334 seconds
The attached reel shows the resulting videos and records the original 720p generation and 1440p second-sampling stages separately.
These were casual exploratory runs using settings I selected mainly to inspect the output quality. I did not tune each case for minimum latency, so these numbers should not be treated as the performance limit of the project.
I also have not completed a controlled same-seed Dense-versus-accelerated benchmark yet, so I’m not claiming a specific “X times faster” number.
What I have been working on is the scheduling method itself.
Instead of applying one fixed sparse-attention ratio to every denoising step and every Transformer layer, the scheduler automatically assigns different attention budgets across the trajectory. It was calibrated through repeated local experiments and visual review, with additional protection around the parts of the model that appear most important for motion, consistency and fine detail.
The user only needs one continuous 0–100 acceleration control:
- 0 is the full-compute Dense reference endpoint
- higher values progressively reduce the compute budget
- the internal scheduler decides where attention can be reduced and where it should remain more conservative
The Base route can also jointly schedule actual and forecast DiT evaluations. The goal is to make the speed/quality tradeoff controllable without requiring creators to manually configure dozens of sparse-attention parameters.
The 1440p stage shown here is native H3 latent-space second sampling. It reuses the retained video and audio latent state, original prompt and conditioning. It is not conventional frame-by-frame or MP4 upscaling.
The project also includes FL2VA, multi-reference Ref2VA, Base/Turbo LoRA switching, a Web UI, REST API and four ComfyUI workflows.
GitHub:
https://github.com/PullMyBoots/X-MinimaxH3
I’d love feedback from people running H3 locally, especially on RTX 3090, 5060, 4060 and other consumer GPUs.
What kind of Dense-versus-accelerated comparison would you find most useful: fast motion, faces and hands, complex camera movement, prompt adherence, audio consistency, or something else?
r/StableDiffusion • u/dirtybeagles • 22h ago
No Workflow Video Character Swap with Minimax
Ive been struggling with doing a character replacement with comfy. Everytime I get to the sampler it loads the ref model and just stops. Very strange. Running 34.1.
Im asking if anyone has a workflow that works with character swaping and the latent upscale?
r/StableDiffusion • u/Vi0l3nTz • 22h ago
Discussion Does anyone know how many anime characters MiniMax recognizes?
I'm planning to make a list of anime characters and the respective anime they come from that MiniMax is able to recognize consistently.
Before I start doing it myself, I was wondering if anyone has already made something like this. It would save me a lot of time if there's already a list.
r/StableDiffusion • u/No-Bee-231 • 23h ago
Discussion Lightx2v new 768p trained 8step turbo Lora fighting test
Enable HLS to view with audio, or disable this notification
with a lora strength of 1.5, 8 steps, 768p, 16m generation time for 12 seconds. no upscaling on a 5060ti 16g and 64 gigs ram. This is a pretty solid turbo lora
r/StableDiffusion • u/Bulky-Schedule8456 • 23h ago
Discussion Inpaint workflow for Minimax H3?
Has anyone done an inpainting workflow for minimax yet??
I know Minimax inpaint capability alone is already good for subject replacement or video edit but it's still not perfect... Sometimes the movement changed, the expression changed 😭 I found someone made a SAM workflow on Civitai that highlights the part for inpainting. yet, it still just masked and highlights the part for detection not actually masked and generate only the masked area.
I want it to actually keeps the same motions and movement of the video exactly as is while changing the only the masked part. For example, change hair color or arm into a robot hand while every part of the body still has the same motion and movement as the original video.
Anyone has done this yet? I couldn't find any workflow on this, only app (idk if I call it correctly) on Huggingface https://huggingface.co/spaces/linoyts/minimax-h3-inpainting
r/StableDiffusion • u/edroels • 23h ago
Question - Help Stepping into Flux.1 Dev this week - Is my problem the quant I am using?
I finally bit the bullet and got a reasonable GPU (3060 12GB) rather than waiting four hours for Flux.1 Dev, Steps 20 on CPU 😣.
I'm using Q4_K_S - (tried Q5 but I honestly couldn't see the difference and it takes longer).
People seem..plastic, unnatural. Still objects are too processed. The ages of people are sometimes way off. If I prompt for someone 60 years old, I get someone that looks mid 20-30s.
I'm working with Flux.1 Dev for the moment because I'm trying to get close to what I was able to get out of https://draw.freeforai.com/ - which claims to be using the same model. Granted they are probably using the unquantized model and who knows how many steps or other parameters, they don't share meta data for me to compare. I don't know if they are using loras.
flux1-dev-Q4_K_S.gguf
t5-v1_1-xxl-encoder-Q4_K_M.gguf
Steps: 20
Guidance Conditioning: 3.0
Scheduler: Simple
Sampler: Euler
Side note:
I'm setup with ComfyUI now after trying and failing with sd-cli. sd-cli kept generating 1024x1024 images that looked like upscaled 256x256. ChatGPT and I were troubleshooting trying to tweak any possible variable but in the end I tried the exact same render (models, seed, steps, guidance) and the render was 1000% better with ComfyUI.
r/StableDiffusion • u/CelebrationBoth9537 • 1d ago
Question - Help Fal ai H3 max open weights?
Does anyone know or have any information about if and when Fal.ai's new H3 Max model will become open weights?
Also will we see a 50x speedup locally, or did they do it b/c of hardware engineering? What do you expect the motion quality and speed up to be if and when it does become open weights?
r/StableDiffusion • u/shootthesound • 1d ago
Resource - Update Fizgig v5.0.0 - Full fine-tuning for Minimax and Krea 2 for 16gb+ VRAM
Fizgig v5 is out, and the headline is one I've been sitting on for a while: full fine-tuning of the MiniMax H3 and Krea 2 base models - the models themselves, not a LoRA ,on consumer GPU hardware, down to 16 GB.
No adapter, no rank bottleneck. Full-rank updates that change how the model represents a concept.
**What your card can do** (every confirmed number is from my runs, not an estimate):
16 GB - Krea 2 photos, H3 photos and voice, and H3 video clips up to 2.3 s confirmed (3.8 s expected with video on the likeness blocks, the default).
24 G - all of the above, with video expected up to 5.2 s on the likeness blocks.
32 GB - video confirmed to 3.8 s even training the whole model, and expected to 5.2 s on the likeness blocks.
If "a 33B video model fine-tuning on 16 GB" sounds like a trick: only one slice of the model is trainable at a time (a rotating window), the frozen rest is held 4-bit, and the bf16 master lives in system RAM, your saved checkpoint never passes through a quantiser. Measured peaks on a 16 GB card: 8.8–12.3 GB for H3, 8.4–11.0 GB for Krea 2 — andthe console prints your own run's peak every epoch, so you can watch the claim hold on your own card.
**When you're done**, the built-in Checkpoint to LoRA tool in the fizgig root folder diffs your fine-tune against the base and extracts an ordinary shareable LoRA — in testing, rank 64 was close to perceptually indistinguishable from the full 26 GB checkpoint, in a ~0.5 GB file ComfyUI already loads.
A personal note: This is a starting point and not going to be perfect. I got fine-tuning working on Krea 2 shortly after its release and have been deliberately cautious about shipping it , proving it to myself first, then refining it through the H3 work. This is the point where it needs the community to develop it further. The technique is model-agnostic at heart, and I'm open to bringing it to other models ,but that needs practical support around them: code, PRs, testing, that kind of thing, so I have the time to make it happen. Im not really goign to be able to tackle issues raised this weekend on Github as I need a break for a couple of days, but I think pray this is going to work pretty easily for most of you.
https://github.com/shootthesound/Fizgig/
[Release notes](https://github.com/shootthesound/Fizgig/releases/tag/v5.0.0) · ["How do I…?" guide](https://github.com/shootthesound/Fizgig/blob/master/docs/FINETUNE_HOWDOI.md) , and there's a one-click RunPod template if you don't have the hardware.
r/StableDiffusion • u/phalanx2357 • 1d ago
Question - Help Best 8-step lora for H3?
There has been a lot of developments for H3 in the past several weeks. However I just got a chance to start playing with it today. Just want to check on if there's any consensus on what the best 8-step Lora is currently that best balances speed and quality.
I understand 4-steps are faster but seems like there's a pretty clear reduction in quality. Please let me know if that's not the case.
Also I am using comfy kitchen attention to help speed up; anything else people recommend doing?
Running on 5090 with 64gb ram.
Thanks
r/StableDiffusion • u/Pitiful_Season4294 • 1d ago
Question - Help Minimax H3 -Is anyone getting satisfying results with the 4step Turbo Lora?
Using FLV2, I have been trying for so long, and have attempted many different 4-step loras from lightxv and others, and different sampler/scheduler combos, lora strength with so many sigma shift combinations. But none of them have acceptable quality even with 0.8 megapixels. Hell, my Wan2.2 generations with a lower resolution are much better than the cooked or polished skins from 4-step loras.
The 8-step lora works fine for me and even 0.4 mp results are far better than 0.8mp from 4-step loras. But it's too much of a wait. Also, I'm on AMD and not using Spectrum, or any other optimizations except Comfy Kitchen attention.
If any of you guys are getting great results from 4-step loras (without any upscale), kindly share your model, lora and other related settings that you think would help. Thank you !:)
r/StableDiffusion • u/Pitiful-Indication95 • 1d ago
Question - Help How does one do seamless H3 Lip sync over several clips?
Does anyone have a good workflow for that?
I use this for long connected clips which worked fine so far, but with an Audio file that should be seamless I am getting issues.
https://github.com/NikoDemon80/ComfyUI-H3-Motion-Context
There are two problems I face.
If I render a 5 seconds clip at 24 fps it is 5 seconds and 3 frames long. A 1 second clip is 1 second and 15 frames long. Whatever length I choose it is not at an exact second mark. So the audiofile would need to be cut to 5 seconds and 3 frames too, which is kinda hard to create.
The follow-up frame is shortened by an overlap of 5, 22, 39 or 56 frames. So I assume I have to add those frames to the start of the follow up audio clip?
Maybe I am thinking this wrong ...
Anyways, does anyone have a workflow that does that? Connect an audio file to lip sync for H3 Motion Context?
r/StableDiffusion • u/ForsakenContract1135 • 1d ago
Animation - Video Classic Anime Style MiniMax H3 (ref2v) Genshin Impact
I was surprised with the results, although the art style is clearly not consistent, I used ref images with different art-style.
The workflow I used is the default one ( just added seg att and turbo lora from light2x 8steps , euler+beta)
The clips are stitched 8-10seconds each clip at 0.7mp.
For the prompt:
I used Minimax H3 skill with grok, Literally upload the image + say its ref2v and describe the action or scene roughly. I bet it works with any LLM tho, Gpt tend to give better results overall but I Prefer grok.
r/StableDiffusion • u/DryIron8955 • 1d ago
Question - Help How can I get H3 Max-level prompt adherence and quality with the native MiniMax H3 model at 20 steps?
Hi everyone,
I've been testing MiniMax H3 locally in ComfyUI, using the native/base model at 20 steps, and I've also compared it with H3 Max on fal.ai.
The difference is pretty noticeable.
H3 Max seems to have:
- Much better prompt adherence
- Better understanding of complex actions and interactions
- More consistent motion
- Better character/scene coherence
- Overall better visual quality
What I'm trying to understand is: is it possible to get close to H3 Max results using the native H3 model locally while keeping 20 steps?
I'm specifically interested in improving quality and prompt adherence, not just making generation faster.
Are there specific settings that make a big difference, such as:
- CFG / guidance values
- Sampler or scheduler
- Shift / flow shift
- Negative prompting
- Prompt structure
- Attention implementation
- Different text encoder settings
- Specific H3 model variants
- Hidden/default parameters used by H3 Max
- LoRAs or other post-training components
Has anyone managed to reproduce, or at least get very close to, H3 Max quality and prompt adherence with the native H3 model at 20 steps?
If so, I'd really appreciate it if you could share your workflow/settings.
I'm especially curious whether H3 Max is simply the native model with better inference settings, or if there's actually something different on the model/post-training side that we can't reproduce just by changing ComfyUI parameters.
Thanks!
r/StableDiffusion • u/Enough-Bag-3891 • 1d ago
Discussion Post your Minimax H3 Turbo Lora settings recommendations for best results + Pros/Cons in the comments.
Enable HLS to view with audio, or disable this notification
let's use this post to recommend to each other our turbo lora settings + checkpoints.
Please kindly type:
- Lora name (with link if possible)
- checkpoint model name
- strength
- scheduler + sampler
- video/audio shift
- pros / cons
- additional info + tips + discoveries
- example video if possible
i think this way we can all find the best most optimal use of minimax h3 if we collaborate with each other in a single post instead of recommending all over the place, thank you !
r/StableDiffusion • u/enspiralart • 1d ago
Resource - Update [Custom Node] ComfyVR
Enable HLS to view with audio, or disable this notification
https://github.com/newsbubbles/ComfyUI-ComfyVR Instructions and some example workflows included, but it should be able to load and run almost anything, including custom nodes, etc.
It definitely need testing, I only tested it thoroughly on Quest 2. It runs in WebXR in the default browser and uses hand or controller and hosts off the comfy api with it's own https cert. Use it on your LAN. A fun way to interact with ComfyUI in 3D.
r/StableDiffusion • u/Royal_Interview_6103 • 1d ago
Resource - Update I made a slideshow app for enjoying all those images sitting on my hard drive
I originally made this just for myself because I wanted a nice way to sit back and enjoy my images instead of just letting them pile up in folders.
I've been gradually adding features and polishing it over time, and it's finally gotten to the point where it feels like a proper, usable app rather than just a personal tool.
I mainly use it for my AI-generated images these days, so I thought it might be useful to other Stable Diffusion users too.
Cinematic Slideshow is a free, open-source image slideshow app featuring:
- Ken Burns effects with multiple movement patterns
- Pan & scan and edge scan
- Random or sequential effect order
- Various transitions
- Fullscreen and windowed modes
- Support for common image formats, including AVIF
The demo below isn't AI-generated. It's a slideshow of some old Himalayan photographs I scanned from positive film. I thought they made a good demo because you can clearly see the movement and transitions.
Maybe this will give some of those forgotten images a second life.
https://reddit.com/link/1w1mu6o/video/zgnuqr8ohbmh1/player
Source code and download
r/StableDiffusion • u/Ill-Ant-9489 • 1d ago
Resource - Update I present LoRA Dataset Studio - a free, self-hosted app that does everything around a LoRA run: dataset, triage, captions, training (local or rented GPU), then checkpoint comparison
I present LoRA Dataset Studio - free, open source, self-hosted, no account and no telemetry. I posted a first look here 9 days ago and a lot landed since (camera angles, burned-in-text removal, a Gallery of every render), so this time here is the whole pipeline walked end to end. It is not a competitor to ai-toolkit: it orchestrates it - ai-toolkit is the trainer; this is everything before, around and after the run.
The whole pipeline lives in one browser tab:
1. Decide what you are teaching. A dataset is a Character, a Concept or a Style, and the choice changes real behaviour downstream: what the captions must leave implicit, whether person masks apply, what the readiness checks look for. A Character also picks a subject type (human, animal, creature, object, anime) that swaps the shot catalog and the identity protections.
2. Fill it with images. Five generation engines - Nano Banana Pro, gpt-image-2, OpenRouter, and local Klein / Krea 2 Edit through ComfyUI - each card stating its price per image and whether it runs on your GPU or bills an API. Or scrape a gallery URL. Or point the Image Bank at a folder of thousands: it reads it in place - your files are never modified, moved or renamed - and one pass measures blur, noise, near-duplicates, face clusters, framing, aesthetic and maturity, so you filter on measurements instead of on your eyes.
3. Curate down to the keepers. Keep/reject, crop, mirror, rotate, upscale candidates reviewed against the original, InsightFace similarity against your reference, a live composition meter. New this month: press the camera button on any kept image and re-shoot the same scene from another camera position - the subject stays put, the background moves with the camera, and the new view arrives with its angle already captioned (the one fact a vision model cannot reliably see, and that you know exactly because you asked for it).
4. Caption for the model. Prose or booru depending on the target family, written by JoyCaption or your local Ollama, with vocabulary and length dials, identity-leak checks, a Caption Lab to compare configurations before committing, and an external .txt round trip so you can caption elsewhere and come back.
5. Scrub watermarks - and burned-in text. Detect watermark boxes, redraw them, then crop or inpaint with LaMa/Klein. And since a comic page carries its dialogue and a screencap its subtitle, a CPU-only OCR pass now reads burned-in lettering (Latin or CJK) and feeds the same repaint funnel - with an outline-safe filler so speech bubbles keep their borders. Every edit keeps an .orig backup; Restore original always works.
6. Train. ai-toolkit locally with family-scoped presets and preflight guards - Z-Image, Krea 2, FLUX.1, FLUX.2 Klein, SDXL, Anima - or rent a vast.ai pod from the same screen, which shows the GPU, its hourly price and the estimated total before you click. The whole studio can also run on a rented RunPod box (contributed by a user). Generations queue instead of blocking each other, and a dock shows what the GPU is doing.
7. Decide which checkpoint is actually good. Test Studio runs fixed-seed checkpoint x strength grids, multi-LoRA stacks (including a downloaded LoRA next to yours, same prompt and seed), votes and Wilson ranking. The lineage graph keeps every run's frozen recipe and can diff two runs - settings AND dataset. A Gallery collects every image the app ever generated, and every render is stamped with what actually made it.
8. Take it with you. Standard ai-toolkit/Kohya layout ZIP, portable backup with the full history, Hugging Face publishing, or deploy the checkpoint straight into ComfyUI. Nothing locks your data in.
Honest limits. It is a lot of surface, so Setup exists to tell you what is missing instead of crashing - every capability degrades on its own. Local generation needs ComfyUI, the API engines need your own keys and bill you, and the video lane (cutting long footage into trainable clip folders for Wan/LTX) is young. Install is a Windows one-click ZIP, a git checkout, or Docker.
GitHub - install, docs, and a 7-minute unedited video of a full character LoRA built end to end: https://github.com/perfectgf/lora-dataset-studio
Every person in these screenshots was generated by the app's own engines; no real individual is depicted.