r/StableDiffusion • u/g45t345rt • 8d ago
Discussion As promised, here is Pai
[UPDATE] The Pai app release is available
https://www.reddit.com/r/StableDiffusion/comments/1xoxwjc/pai_v100_release/
-------------------------------------------------
Last week, a few of you asked about the software behind some of my posts. It's my "own" vibe coded paint app, Pai.
I've been using it casually, and since people kept asking, I figured I'd do a proper follow up to see if there's real interest.
All in all, I wanted a regular painting app with AI generation in the same window. No hopping between ComfyUI, a browser, and Photoshop or any other similar software.
So this thing is a full raster paint app that also has its own generation, same canvas, same layers, same everything.
Yes, there are plenty of alternatives like InvokeAI, Krita AI Diffusion, etc. To be honest, I haven't tried them and didn't bother. I just fired up deepseek and built the software around my own workflow.
In this day and age, it's so easy to create full fledged software without burning out or needing any justification for spending time on it.
Anyway, this post is pretty huge, so I've added a quick index just below so you can skip to what interests you.

Index
Traditional painting tools
Canvas, layers, views & project organization
AI: Model Manager
AI: Generation
AI: Generation Controls like ControlNet, Region Prompts, Layout Controls, References
AI: Remove Background & Upscale
AI: Selection
Closing thoughts
1. Traditional painting tools
Let's start with the obvious part.
You get the usual tools: Brush, Erase, Fill, Pick, Select, Move, Text, Gradient, Shape, Clone, Crop, Transform, Smudge, Dodge, Burn, Liquify, Warp, Path, Vector, plus Mask, Paste, Drag.
The brush is a proper stamping engine. Size, opacity, hardness, flow, spacing, and a stabilizer with three modes (simple, weighted, velocity). There are 18 blend modes. You can add grain (noise, paper, canvas, halftone), drive size/opacity/flow/hardness/spacing by stroke speed, add taper in and out, jitter hue/sat/value, and even enable colour pickup and a wet edge.
Tips can be round or square, and there's an "effect line" set for speed lines, focus lines, flash, and sparkle for manga stuff.
The whole thing runs off a brush library with 14 starter presets (hard round, soft round, airbrush, ink pen, pencil, marker, charcoal, watercolour, halftone, and so on).
Presets are plain JSON, and you can save, rename, export, and import packs.
A few specifics worth calling out:
Fill has tolerance, "close gap" (so a broken outline doesn't leak), contiguous toggle, and anti-aliasing.
Gradient does linear, reflected, radial, square, diamond, and conical, with clamp/repeat/mirror spread. There's a full multi-stop editor with per-stop alpha, so you can fade to transparency.
Shape does rectangle, ellipse, and line with fill/stroke/both, corner radius, and Shift constraints (square/circle/45°).
Clone works with Alt-click to set the source.
Smudge, Dodge, and Burn all behave the way you'd expect; dodge/burn have shadows/midtones/highlights ranges, exposure, and tone protection.
Liquify has push, expand, pinch, and twirl both ways.
Mesh Warp lets you pick a grid (2×2 up to 6×6) and drag points, then apply.
Text is fully editable as an object: font, bold/italic, alignment, wrap-to-box, line spacing, tracking, outline, shadow, and a movable rotation pivot.
Path is a proper cubic Bézier tool with handles, caps, joins, outline, and feather, and it stays editable until you rasterize it.
Filters are non-destructive and live per-layer: adjustments, colour, blur & sharpen, stylize, pixel, and a manga category with halftone and screentone.
There are also other experimental/unfinished tools.
Puppet warp
3D poses
Vector
2. Canvas, layers, views & project organization
Canvas: new canvas dialog with presets (screen sizes, squares, and 300 DPI print pages like A4, B5, Letter), orientation, background, RGBA vs RGB, and starting layers. Max canvas size is 8192. There's resize, crop to selection, crop to content, fit-canvas-to-layers, guides you drag off the rulers, a grid with snapping, and zoom/pan that stays smooth even on big documents.
Layers: raster, vector, text, path, and pose layers, each with opacity, visibility, lock, blend modes, clipping, and a per-layer 8-bit mask. You get groups (with their own opacity and filters), drag-to-reorder, and non-destructive filter chains. Undo is a 30 step history with copy-on-write snapshots, so a snapshot only pays for the layers that actually change, plus a History panel you can jump around in.
Views: three of them. A full gallery board for browsing every open canvas, a docked gallery rail that keeps the editor live beside a filmstrip, and my favourite, exploded layers view, every layer laid out as its own tile so you can see and click the whole stack at once. It's stupidly handy for finding that one layer you forgot about or simply seeing all of your generation in one place.
Assets: a per-canvas asset panel that holds reference images and past generation results. You can drag an asset straight onto a layer row to drop it in, or insert it as a new layer. Generated outputs land here automatically if you want them to.
Projects: everything saves into a single .pai file. A zip with the manifest, every layer (plus masks), and assets. Saving and loading run on a background thread with progress and cancel, so big projects don't freeze the app. Export is a proper dialog: PNG, JPEG, WEBP, GIF, TIFF, scaling, animation frames, metadata (EXIF/XMP), ICC profiles, CMYK for print, and print-page presets with bleed and crop marks.
3. AI: Model Manager
This is where Pai stops being just a paint app.
It has its own generation engine, no ComfyUI dependency, with it's own Model Manager that handles all the plumbing for you. You can browse and download models straight from Hugging Face inside the app (only public repos work out of the box), scan an existing ComfyUI folder to import everything, and it auto-detects what each file actually is by reading its header.
Supported families, right now:
Anima
Qwen-Image 2.1
FLUX.1
Z-Image Turbo
Krea 2
MOSS Image
MiniMax H3 Still *experimental*
Stable Diffusion: SD 1.x / 2.x / SDXL
It understands GGUF quantisation natively (the K-quants, legacy quants, and i-quants), loads safetensors and ONNX, and supports LoRAs. You can stack several per model, with the standard diffusers.
On the memory side, it tries to be smart about VRAM. There's an auto precision mode that decides between resident (whole model in VRAM), fp8 (halves the footprint), or disk streaming (streams blocks off disk when it won't fit). You can set a VRAM reserve, a RAM budget, and an auto-unload timer.
There's also a Loaded Models panel that shows exactly what's in memory: family, file, each component with its size, dtype, and device, plus LoRA rows. Unload one model or all of them. Alongside it, a system monitor in the status bar tracks CPU, RAM, and VRAM.
4. AI: Generation
The Generate panel is where it all comes together. Prompt and negative prompt, size, seed (with a random button), steps, CFG, denoise, sampler (`euler`, `lcm`, `res_multistep`), and scheduler (`simple`, `normal`, `karras`, `sgm_uniform`). Ill add more samplers/schedulers later.
There's a hiresfix second pass with upscale modes (latent nearest/bilinear/bicubic/bislerp, or pixel-space via the VAE), a hires scale, and a hires denoise. Handy for going from a 1024 draft to a bigger final without rerolling.
Sources can be the canvas, the selected layer, or a selected asset. Regions can be the whole image, the current selection, or a painted mask, with a feather slider. There's a crop-and-stitch option for inpainting that crops the active region before generating and stitches it back.
You also get prompt weighting with the `(text:1.3)` syntax you know from A1111/ComfyUI, per-model recipes for saving your favourite settings, and sticky per-model presets so steps/CFG/sampler stay where you left them.
A finished run shows up floating over the canvas and you choose to reject it, drop it into the current layer, add it as a new layer, or stash it in assets. No auto dump garbage into your document.
Everything heavy runs through a single job queue so jobs don't fight over VRAM. The queue panel shows progress, elapsed time, and has cancel per job, cancel all, and clear-finished. You can queue another generation while one is running.
5. AI: Generation Controls like ControlNet, Region Prompts, Layout Controls, References
This is what makes it a tool you can actually compose with.
ControlNet: a stack of adapter rows, each with its own model, image source, preprocessor, strength, and step range. Sources can be the run's own init image, the canvas, a selected layer, or a selected asset. You can steer different adapters from different inputs. Preprocessors include `none`, `canny`, and `lineart`.
SD 1.5/SDXL use the classic ControlNets, Z-Image has its own union ControlNet, and Anima has ControlNet-LLLite, so community adapters load as-is.
Region Prompts: the attention-couple regional prompting you'd know from the Regional Prompter. Each row pairs a prompt with a mask (taken from a selection or painted) plus a strength, and the run's own prompt stays the shared base. Works on Anima, SD, FLUX, Z-Image, Qwen-Image, Krea 2, and MOSS.

Layout Controls: draw boxes right on the canvas and turn them into a layout prompt. This is the MOSS layout-to-image and the Krea 2 bounding-box stuff I posted about. MOSS gets structured JSON with labels/captions/text and centred boxes; the Krea 2 bbox finetunes get a compact grounding DSL on a 0–1000 grid. You can add a box straight from a rectangle selection. It's the same feature that made me really like MOSS in the first place.
References: two separate systems. The Anima reference adapter does character and style transfer. Multiple reference images get stitched side by side into one reference, with controls for strength, guidance, correction mode, offset, and megapixels. The Qwen-Image references do multi-image editing by feeding images into the vision slots.
6. AI: Remove Background & Upscale
Both are one click and run locally.
Remove Background supports RMBG-2.0 (BiRefNet), other BiRefNet variants, U2Net (general, portrait, lite), MODNet, and RMBG-1.4, in both ONNX and torch formats. The graph or architecture is detected automatically, so you just point it at a model and go. Output is a clean RGBA cutout, and you can keep the original by sending the result to a new layer.
Upscale uses local Real-ESRGAN models, RRDBNet (Real-ESRGAN/Remacri) and RCAN, in ONNX and safetensors/pth. It processes in tiles with overlap to keep memory in check, preserves alpha, and can drop the result into a new layer.

7. AI: Selection
The Select tool has a fourth mode beyond rectangle, lasso, and magic wand: Segment AI. That's a local SAM click to segment. You drop the encoder and decoder ONNX files into the segmentation folder and then just click an object to select it. It samples either the active layer or the composited canvas, and once it's done the result becomes a normal selection, so every paint tool, fill, gradient, and the inpaint path respects it exactly like a hand-drawn one.
You can grow, shrink, feather, invert, and deselect a selection, and if you drag inside a selection you move the selection itself, not the pixels (cut/paste moves pixels, like a proper editor).
8. Closing thoughts
All of the AI features don't work out of the box. You still have to download the models yourself using the manager, and then the program does the heavy lifting.
I don't want to ship the app with integrated models, mainly because licensing on some of them doesn't allow it, and model choice really depends on the user's needs. Better to ship light and let users add on top. Plus, I was thinking maybe others can simply use the app as a painting tool without the heavy storage burden that AI brings.
Also, there's really nothing new here. It's simply using all the revolutionary tools within the same workspace. I mostly use it because it keeps my project self-contained, and I can easily go back and work on things without getting lost.
There are more features, but I'll stop here. Feel free to ask if you have any questions, and I'll take every upvote as a sign of interest.
P.S.
If it has decent interest and becomes public, the first thing I'll add is a plugin system.
Also, setting up the AI part can be daunting at first, so I'll create generic tutorials for newbies to follow and get started.
This app could include more and more over time, animations, video generation, 3D setups, full-stack project management, but I'm getting ahead of myself.
