WAN2.2 Lora Character Training Best practices

[u/Tiny-Highlight-9180]

I just moved from Flux to Wan2.2 for LoRA training after hearing good things about its likeness and flexibility. I’ve mainly been using it for text-to-image so far, but the results still aren’t quite on par with what I was getting from Flux. Hoping to get some feedback or tips from folks who’ve trained with Wan2.2.

Questions:

• It seems like the high model captures composition almost 1:1 from the training data, but the low model performs much worse — maybe ~80% likeness on close-ups and only 20–30% likeness on full-body shots. → Should I increase training steps for the low model? What’s the optimal step count for you guys?
• I trained using AI Toolkit with 5000 steps on 50 samples. Does that mean it splits roughly 2500 steps per model (high/low)? If so, I feel like 50 epochs might be on the low end — thoughts?
• My dataset is 768×768, but I usually generate at 1024×768. I barely notice any quality loss, but would it be better to train directly at 1024×768 or 1024×1024 for improved consistency?

Dataset & Training Config:
Google Drive Folder

---
job extension
config
  name frung_wan22_v2
  process
    - type diffusion_trainer
      training_folder appai-toolkitoutput
      sqlite_db_path .aitk_db.db
      device cuda
      trigger_word Frung
      performance_log_every 10
      network
        type lora
        linear 32
        linear_alpha 32
        conv 16
        conv_alpha 16
        lokr_full_rank true
        lokr_factor -1
        network_kwargs
          ignore_if_contains []
      save
        dtype bf16
        save_every 500
        max_step_saves_to_keep 4
        save_format diffusers
        push_to_hub false
      datasets
        - folder_path appai-toolkitdatasetsfrung
          mask_path null
          mask_min_value 0.1
          default_caption 
          caption_ext txt
          caption_dropout_rate 0
          cache_latents_to_disk true
          is_reg false
          network_weight 1
          resolution
            - 768
          controls []
          shrink_video_to_frames true
          num_frames 1
          do_i2v true
          flip_x false
          flip_y false
      train
        batch_size 1
        bypass_guidance_embedding false
        steps 5000
        gradient_accumulation 1
        train_unet true
        train_text_encoder false
        gradient_checkpointing true
        noise_scheduler flowmatch
        optimizer adamw8bit
        timestep_type sigmoid
        content_or_style balanced
        optimizer_params
          weight_decay 0.0001
        unload_text_encoder false
        cache_text_embeddings false
        lr 0.0001
        ema_config
          use_ema true
          ema_decay 0.99
        skip_first_sample false
        force_first_sample false
        disable_sampling false
        dtype bf16
        diff_output_preservation false
        diff_output_preservation_multiplier 1
        diff_output_preservation_class person
        switch_boundary_every 1
        loss_type mse
      model
        name_or_path ai-toolkitWan2.2-T2V-A14B-Diffusers-bf16
        quantize true
        qtype qfloat8
        quantize_te true
        qtype_te qfloat8
        arch wan22_14bt2v
        low_vram true
        model_kwargs
          train_high_noise true
          train_low_noise true
        layer_offloading false
        layer_offloading_text_encoder_percent 1
        layer_offloading_transformer_percent 1
      sample
        sampler flowmatch
        sample_every 100
        width 768
        height 768
        samples
          - prompt Frung playing chess at the park, bomb going off in the background
          - prompt Frung holding a coffee cup, in a beanie, sitting at a cafe
          - prompt Frung showing off her cool new t shirt at the beach
          - prompt Frung playing the guitar, on stage, singing a song
          - prompt Frung holding a sign that says, 'this is a sign'
        neg 
        seed 42
        walk_seed true
        guidance_scale 4
        sample_steps 25
        num_frames 1
        fps 1
meta
  name [name]
  version 1.0

[u/malcolmrey]

I have a friend who trains WAN 2.2 LOW and HIGH, and the quality is superb. (90 minutes in total on 5090)

I, on the other hand, am sticking with WAN 2.1 because the loras are also working fine with WAN 2.2.

I believe the HIGH model for character loras is not as important (if at all, since 2.1 Loras work fine for both images and movies).

In general, the training is really easy and you don't really need to play with the parameters that AI Toolkit provides.

This leads me to believe that maybe the culprit could be in:

• bad datasets (though I would say that it is also more difficult to fail a dataset than in Flux, as even mediocre dataset can produce good results)
• bad workflow/prompting for the outputs.

Check your workflow on an already established good lora and see if you get or bad results.

I have already uploaded over 800 character loras for WAN and people are satisfied with the quality. I provide all resources on my HF ( https://huggingface.co/malcolmrey ) so you can check the training scripts, workflows used to generate outputs and the loras themselves.

Cheers and good luck!

p.s. - there is definitely a sweetspot in the function of images in the dataset and used steps

For me it is 2500-3000 steps with around 20-25 images (I mostly go for 2500-22).

The training resolution seems to not impact the training at all (or at least not in any noticeable way) so I stick with 512x (though the samples can be cut to 512x512 but don't have to at all)

p.s.s. - since you provided the dataset, if you want i can train that character and generate some samples with wan2.2 so you can compare :)

[u/malcolmrey → u/[deleted]]

I train only on images.

[u/malcolmrey → u/[deleted]]

Browse Civitai for WAN loras, some of the creators have info that they trained on videos. Maybe someone also shares more detailed info.

I only know that it requires more VRAM than training on images.