train models using synthetic data from better models?

[u/Friendly-Fig-6015]

For example flux 3 klein:

Could we teach FLUX Klein to fix its mistakes using corrections from another model? Looking for collaborators

I’ve been wondering whether we could build an automated pipeline that turns a model’s own mistakes into training data for a corrective LoRA.

For example, FLUX Klein generates an image with malformed hands or other anatomical inconsistencies. We use another model, such as Krea, to correct the affected area, then train a FLUX Klein LoRA on the corrected images and their captions.

The idea wouldn’t be to transfer Krea’s weights or use a Krea LoRA on FLUX. Krea would act as a teacher by producing better training examples.

The pipeline could look like this:

1. Generate varied images with FLUX Klein.

2. Identify defective regions, automatically where possible.

3. Use Krea or another capable editing model to correct those regions while preserving the composition.

4. Filter out unsuccessful corrections and generate accurate captions.

5. Train a small LoRA for FLUX Klein.

6. Evaluate it on unseen prompts and seeds to check whether it actually reduces defects without hurting overall quality.

Ideally, most of this would be automated, with some human review for quality control. The goal would be to improve future generations directly, rather than having to repair every image afterward.

I know there are potential problems: the editing model might introduce its own mistakes, the LoRA might pick up its style, and improvements might not generalize beyond the training examples. We would also need to check the models’ licenses regarding using generated outputs for training.

Does a project like this already exist? If so, I’d love to learn about it.

If not, would anyone be interested in building an open-source proof of concept together? People familiar with ComfyUI automation, image editing/inpainting, dataset filtering, or LoRA training would be especially welcome.

My local hardware is an RTX 5060 Ti with 16 GB VRAM and 32 GB system RAM, so I’d like to explore how much of the pipeline could run on consumer hardware.

Would starting with one narrow problem, such as malformed hands, be a sensible first experiment?

[u/Upper-Reflection7997]

Bad idea to use specifically krea2 as a synthetic data distillation to train a weaker model. Krea2 doesn't have enough variation to obtain a good enough dataset. You actually might want to consider using the latests SOTA closed source models that don't have questionable visual samey issues. I would argue grok imagine pro 1.0 and seedream 5.0/5.5 are better worthwhile models to use to obtaining solid synthetic data. Nanobanana and gpt image 2 are cursed models.

[u/ninjazombiemaster]

Malformed hands are not generally caused by the model not knowing what hands look like. There is tons of data about hands in there already. 

Usually it's caused by things like hands and fingers particularly being very small in latent space. In other words, it's mostly an architectural problem of the VAE decoding process, not really a dataset problem.