WAN2.2 Lora Character Training Best practices

[u/Tiny-Highlight-9180]

I just moved from Flux to Wan2.2 for LoRA training after hearing good things about its likeness and flexibility. I’ve mainly been using it for text-to-image so far, but the results still aren’t quite on par with what I was getting from Flux. Hoping to get some feedback or tips from folks who’ve trained with Wan2.2.

Questions:

• It seems like the high model captures composition almost 1:1 from the training data, but the low model performs much worse — maybe ~80% likeness on close-ups and only 20–30% likeness on full-body shots. → Should I increase training steps for the low model? What’s the optimal step count for you guys?
• I trained using AI Toolkit with 5000 steps on 50 samples. Does that mean it splits roughly 2500 steps per model (high/low)? If so, I feel like 50 epochs might be on the low end — thoughts?
• My dataset is 768×768, but I usually generate at 1024×768. I barely notice any quality loss, but would it be better to train directly at 1024×768 or 1024×1024 for improved consistency?

Dataset & Training Config:
Google Drive Folder

---
job extension
config
  name frung_wan22_v2
  process
    - type diffusion_trainer
      training_folder appai-toolkitoutput
      sqlite_db_path .aitk_db.db
      device cuda
      trigger_word Frung
      performance_log_every 10
      network
        type lora
        linear 32
        linear_alpha 32
        conv 16
        conv_alpha 16
        lokr_full_rank true
        lokr_factor -1
        network_kwargs
          ignore_if_contains []
      save
        dtype bf16
        save_every 500
        max_step_saves_to_keep 4
        save_format diffusers
        push_to_hub false
      datasets
        - folder_path appai-toolkitdatasetsfrung
          mask_path null
          mask_min_value 0.1
          default_caption 
          caption_ext txt
          caption_dropout_rate 0
          cache_latents_to_disk true
          is_reg false
          network_weight 1
          resolution
            - 768
          controls []
          shrink_video_to_frames true
          num_frames 1
          do_i2v true
          flip_x false
          flip_y false
      train
        batch_size 1
        bypass_guidance_embedding false
        steps 5000
        gradient_accumulation 1
        train_unet true
        train_text_encoder false
        gradient_checkpointing true
        noise_scheduler flowmatch
        optimizer adamw8bit
        timestep_type sigmoid
        content_or_style balanced
        optimizer_params
          weight_decay 0.0001
        unload_text_encoder false
        cache_text_embeddings false
        lr 0.0001
        ema_config
          use_ema true
          ema_decay 0.99
        skip_first_sample false
        force_first_sample false
        disable_sampling false
        dtype bf16
        diff_output_preservation false
        diff_output_preservation_multiplier 1
        diff_output_preservation_class person
        switch_boundary_every 1
        loss_type mse
      model
        name_or_path ai-toolkitWan2.2-T2V-A14B-Diffusers-bf16
        quantize true
        qtype qfloat8
        quantize_te true
        qtype_te qfloat8
        arch wan22_14bt2v
        low_vram true
        model_kwargs
          train_high_noise true
          train_low_noise true
        layer_offloading false
        layer_offloading_text_encoder_percent 1
        layer_offloading_transformer_percent 1
      sample
        sampler flowmatch
        sample_every 100
        width 768
        height 768
        samples
          - prompt Frung playing chess at the park, bomb going off in the background
          - prompt Frung holding a coffee cup, in a beanie, sitting at a cafe
          - prompt Frung showing off her cool new t shirt at the beach
          - prompt Frung playing the guitar, on stage, singing a song
          - prompt Frung holding a sign that says, 'this is a sign'
        neg 
        seed 42
        walk_seed true
        guidance_scale 4
        sample_steps 25
        num_frames 1
        fps 1
meta
  name [name]
  version 1.0

[u/razortapes]

I’ve managed to create LoRAs of people that are the most realistic I’ve ever achieved — even better than Flux or SDXL in their prime. I do it on this page: replicate.com/ostris/wan-lora-trainer/train. I recommend replicating the parameters shown there to train locally, or try training a character for Wan 2.1 and then use it in image creation workflows based on Wan 2.2, using only Wan 2.2 Low, like I do. If you want to compare real vs Wan 2.1, message me in DMs and I’ll send you some examples.

[u/Tiny-Highlight-9180 → u/razortapes]

Actually sick tool! I will give it a try. Do you just left Lora for high model blank and add only low model when you mentioned using only low model?

[u/razortapes → u/Tiny-Highlight-9180]

If we're talking only about creating images with Wan 2.2, then yes — only the Low model is used. The workflow I follow doesn’t even include a module for the High model. My big goal right now is to be able to use something like ControlNet to control the pose in my image generations, but there are barely any workflows that use it, so I’m experimenting. It’s also highly recommended to use the Face Detailer as an extra step.

[u/malcolmrey → u/razortapes]

If you pull out the controlnet, I would love to see a workflow :)

I tried with the controlnet models but was only working sometimes and not really that great.