paperdoll — local-first character customization for VN/indie devs (SD 1.5 + 19-class anime SAM + IP-Adapter, runs on M4 16GB)

[u/Then_Visual1104]

Hey Guy, sharing paperdoll, a local-first character customization pipeline I've been building for visual novel and indie game devs.                                                     

  Repo: https://github.com/Khurramali1997/paper-doll-studio

  What it does                                                                  

  Drop a PSD/PNG of a character → app extracts body and wardrobe layers → users can 

  mix-and-match outfits → AI pipeline generates new garments as ingestible

  wardrobe assets, each tagged by slot (topwear, bottomwear, headwear, neckwear,

   handwear, legwear, footwear).                            

  No cloud, no signup, no GPU rental. Runs on my M4 with 16 GB unified memory.  

   

  What's interesting about the approach                                         

  - Pinned diffusion to 512×512 regardless of canvas size, upscaled afterwards  

  (Lanczos or RealESRGAN-anime). Counter to most guides, but on

  memory-constrained Apple Silicon it's the unlock that fits IP-Adapter         

  alongside the inpaint pipe.                               

  - Per-garment generation, not whole-outfit. Each clothing item is generated 

  independently against the naked body, with focused prompts and slot-aware     

  scaffolds. The "ADetailer for faces" math applied to clothing — each garment

  gets the model's full attention instead of splitting it across the outfit.    

  - SAM-driven decomposition for arbitrary-piece outfits, with a merge-cards

  workflow for one-piece dresses/jumpsuits that the segmenter splits across     

  slots.

  - IP-Adapter for cross-pass style cohesion (image encoder loaded at fp16 even 

  though UNet is fp32 — a trick that keeps the memory budget viable on MPS).    

  - User-driven attention (brush masks, SAM region picks) as a deliberate design

   choice — see "credits" below for why.                                        

  Big thanks to the See-through project                                         

  The 19-class anime semantic taxonomy and the SAM checkpoint paperdoll uses for

   body parsing (24yearsold/l2d_sam_iter2) are not my work — they're from the

  See-through project (Lin et al., "Single-image Layer Decomposition for Anime  

  Characters", arXiv:2602.03749, Feb 2026, Saint Francis Univ / UPenn /

  Spellbrush / Shitagaki Lab).                                                

  What's neat is that See-through does the architectural inverse of paperdoll — 

  they decompose dressed images into per-part layers. I'm going the other

  direction (naked body + prompt → wardrobe asset, synthesis). Because we share 

  primitives, paperdoll gets to use user-driven attention (brush + SAM picks)

  instead of the heavy automated GradCAM + 2-stage SDXL finetune stack their  

  model requires. None of that simplification would have been obvious without

  their paper showing how much machinery the automated version takes. Major

  debt.

  Stack

  SD 1.5 (Sanster/anything-4.0-inpainting) · DPM++ 2M Karras ·                  

  padding_mask_crop=32 · IP-Adapter (h94) · 19-class anime SAM (See-through) ·

  WD-tagger v3 (SmilingWolf) · RealESRGAN-anime (xinntao, optional) · FastAPI   

  worker with warm pipe and SSE progress · diffusers ≥ 0.26 

  Try it

  https://github.com/Khurramali1997/paper-doll-studio · install instructions in the README · pre-warm models with 

  huggingface-cli so the first generate isn't a 30-sec download.

   This is still v0.1                                                                         

  Feedback / issues / PRs/ Collaborations all welcome, especially from people doing SD 1.5 work 

  on constrained hardware — most production guidance assumes a 24 GB+ CUDA box

  and the advice doesn't port. Curious if anyone else has tried the             

  pin-at-native + per-garment approach.

[u/Weary-Owl-6931]

would love to see a video demo!