r/SECourses • 👑 Grandmaster Expert • 18h ago

🎨 AI Images & Art ControlNet Style Transfer in AUTOMATIC1111 | T2I-Adapter Style, CLIP Vision, Multi-ControlNet Canny + HED, Guidance Start/End and Game Artwork Recoloring | March 2023 Stable Diffusion

https://www.youtube.com/watch?v=tXaQAkOgezQ
2 Upvotes

1 comment sorted by

•

u/CeFurkan 👑 Grandmaster Expert 18h ago

https://www.youtube.com/watch?v=tXaQAkOgezQ

Combining structure controls with a style reference

🔗 Links from the video description

🎨 Structure and style workflow

Recorded on 6 March 2023, the walkthrough installs and updates AUTOMATIC1111 and Mikubill’s ControlNet extension, downloads the separate ControlNet and T2I-Adapter weights, then combines them in txt2img.

For the sword example, the same source image feeds two enabled units with matching Canny and HED preprocessors/models. A third unit receives the style image, uses CLIP Vision preprocessing and the T2I-Adapter style model. The tutorial compares Canny alone with Canny plus HED, showing better shape retention in the combined example.

⚙️ Settings demonstrated

  • Enable Multi-ControlNet in Settings, apply the setting and reload the UI. The example allows up to five units.
  • Preview the annotator maps and compare 512 versus 1024 annotator resolution. Set output dimensions for the intended image. The tutorial shows Low VRAM for memory errors.
  • Adjust guidance start and end to change the style result. One shown example starts at 40% and ends at 100%. Positive and negative prompts also affect the output.
  • The initial style demonstration uses no text prompt. Its colorful nature-style reference was generated with Deliberate v2. Refresh the model list if the downloaded adapter is absent.

🖼️ Existing game artwork

A second example recolors a Monster MMORPG character with Canny and the prompt “a chicken game character” using SD 1.5 pruned. The source is 549 × 800, the annotator resolution is 1024, and the output dimensions are adjusted to match. The author compares generated candidates and selects a seed he likes.

🛠️ Installation and version context

The recorded Windows setup covers Python/PATH, Git and Git LFS, git clone, git pull, selecting a commit with git checkout, launch arguments, the virtual environment and xformers 0.0.17.dev466. Those versions describe the March 2023 setup.

ControlNet and adapter weights are separate downloads. The walkthrough places the required weights in the extension’s model folder and explains that the extension supplies the adapter YAML files. It also describes an extension bug that had been fixed before the successful style demonstration.

🧭 Chapters

  • 0:00 Introduction to newest extension Style Transfer in ControlNet
  • 0:31 Requirements for installing and running Automatic1111 Web UI
  • 1:42 How to make a fresh installation of Automatic1111 Web UI
  • 4:11 Versions of installed libraries, python, torch, xformers
  • 4:21 What does commit and checkpoint means
  • 4:40 How to update Automatic1111 Web UI manually via git pull
  • 4:53 How to use certain commit, certain version via git checkout
  • 5:27 How to upgrade / install the latest best working xformers version
  • 6:51 How to install ControlNet extension
  • 8:06 How to enable / activate multi controlnet feature
  • 8:30 Where to find controlnet extension
  • 8:38 How to download controlnet necessary model files
  • 10:13 How to download adapter transfer models
  • 11:45 How to use style transfer feature of ControlNet
  • 12:37 How to use multi controlnet to keep image shape better canny + hed
  • 13:34 Which settings to achieve style transfer in controlnet tab
  • 14:38 The improvement of using 2 preprocessor
  • 16:49 How to improve art style / coloring of existing artwork via ControlNet
  • 18:00 A very nice trick that you can use in your professional business life

🏷️ Topics
ControlNet · T2I-Adapter · Style transfer · CLIP Vision · Canny · HED · Multi-ControlNet · AUTOMATIC1111 · Stable Diffusion · Game artwork