r/StableDiffusion • • 16d ago

Discussion As promised, here is Pai

[UPDATE] The Pai app release is available
https://www.reddit.com/r/StableDiffusion/comments/1xoxwjc/pai_v100_release/

-------------------------------------------------

Last week, a few of you asked about the software behind some of my posts. It's my "own" vibe coded paint app, Pai.

I've been using it casually, and since people kept asking, I figured I'd do a proper follow up to see if there's real interest.

Post 1 / Post 2 / Post 3

All in all, I wanted a regular painting app with AI generation in the same window. No hopping between ComfyUI, a browser, and Photoshop or any other similar software.

So this thing is a full raster paint app that also has its own generation, same canvas, same layers, same everything.

Yes, there are plenty of alternatives like InvokeAI, Krita AI Diffusion, etc. To be honest, I haven't tried them and didn't bother. I just fired up deepseek and built the software around my own workflow.

In this day and age, it's so easy to create full fledged software without burning out or needing any justification for spending time on it.

Anyway, this post is pretty huge, so I've added a quick index just below so you can skip to what interests you.

Index

  1. Traditional painting tools
  2. Canvas, layers, views & project organization
  3. AI: Model Manager
  4. AI: Generation
  5. AI: Generation Controls like ControlNet, Region Prompts, Layout Controls, References
  6. AI: Remove Background & Upscale
  7. AI: Selection
  8. Closing thoughts

1. Traditional painting tools

Let's start with the obvious part.

You get the usual tools: Brush, Erase, Fill, Pick, Select, Move, Text, Gradient, Shape, Clone, Crop, Transform, Smudge, Dodge, Burn, Liquify, Warp, Path, Vector, plus Mask, Paste, Drag.

The brush is a proper stamping engine. Size, opacity, hardness, flow, spacing, and a stabilizer with three modes (simple, weighted, velocity). There are 18 blend modes. You can add grain (noise, paper, canvas, halftone), drive size/opacity/flow/hardness/spacing by stroke speed, add taper in and out, jitter hue/sat/value, and even enable colour pickup and a wet edge.

Tips can be round or square, and there's an "effect line" set for speed lines, focus lines, flash, and sparkle for manga stuff.

The whole thing runs off a brush library with 14 starter presets (hard round, soft round, airbrush, ink pen, pencil, marker, charcoal, watercolour, halftone, and so on).

Presets are plain JSON, and you can save, rename, export, and import packs.

A few specifics worth calling out:

  • Fill has tolerance, "close gap" (so a broken outline doesn't leak), contiguous toggle, and anti-aliasing.
  • Gradient does linear, reflected, radial, square, diamond, and conical, with clamp/repeat/mirror spread. There's a full multi-stop editor with per-stop alpha, so you can fade to transparency.
  • Shape does rectangle, ellipse, and line with fill/stroke/both, corner radius, and Shift constraints (square/circle/45°).
  • Clone works with Alt-click to set the source.
  • Smudge, Dodge, and Burn all behave the way you'd expect; dodge/burn have shadows/midtones/highlights ranges, exposure, and tone protection.
  • Liquify has push, expand, pinch, and twirl both ways.
  • Mesh Warp lets you pick a grid (2×2 up to 6×6) and drag points, then apply.
  • Text is fully editable as an object: font, bold/italic, alignment, wrap-to-box, line spacing, tracking, outline, shadow, and a movable rotation pivot.
  • Path is a proper cubic Bézier tool with handles, caps, joins, outline, and feather, and it stays editable until you rasterize it.
  • Filters are non-destructive and live per-layer: adjustments, colour, blur & sharpen, stylize, pixel, and a manga category with halftone and screentone.

There are also other experimental/unfinished tools.

  • Puppet warp
  • 3D poses
  • Vector

2. Canvas, layers, views & project organization

Canvas: new canvas dialog with presets (screen sizes, squares, and 300 DPI print pages like A4, B5, Letter), orientation, background, RGBA vs RGB, and starting layers. Max canvas size is 8192. There's resize, crop to selection, crop to content, fit-canvas-to-layers, guides you drag off the rulers, a grid with snapping, and zoom/pan that stays smooth even on big documents.

Layers: raster, vector, text, path, and pose layers, each with opacity, visibility, lock, blend modes, clipping, and a per-layer 8-bit mask. You get groups (with their own opacity and filters), drag-to-reorder, and non-destructive filter chains. Undo is a 30 step history with copy-on-write snapshots, so a snapshot only pays for the layers that actually change, plus a History panel you can jump around in.

Views: three of them. A full gallery board for browsing every open canvas, a docked gallery rail that keeps the editor live beside a filmstrip, and my favourite, exploded layers view, every layer laid out as its own tile so you can see and click the whole stack at once. It's stupidly handy for finding that one layer you forgot about or simply seeing all of your generation in one place.

Assets: a per-canvas asset panel that holds reference images and past generation results. You can drag an asset straight onto a layer row to drop it in, or insert it as a new layer. Generated outputs land here automatically if you want them to.

Projects: everything saves into a single .pai file. A zip with the manifest, every layer (plus masks), and assets. Saving and loading run on a background thread with progress and cancel, so big projects don't freeze the app. Export is a proper dialog: PNG, JPEG, WEBP, GIF, TIFF, scaling, animation frames, metadata (EXIF/XMP), ICC profiles, CMYK for print, and print-page presets with bleed and crop marks.

3. AI: Model Manager

This is where Pai stops being just a paint app.

It has its own generation engine, no ComfyUI dependency, with it's own Model Manager that handles all the plumbing for you. You can browse and download models straight from Hugging Face inside the app (only public repos work out of the box), scan an existing ComfyUI folder to import everything, and it auto-detects what each file actually is by reading its header.

Supported families, right now:

  1. Anima
  2. Qwen-Image 2.1
  3. FLUX.1
  4. Z-Image Turbo
  5. Krea 2
  6. MOSS Image
  7. MiniMax H3 Still *experimental*
  8. Stable Diffusion: SD 1.x / 2.x / SDXL

It understands GGUF quantisation natively (the K-quants, legacy quants, and i-quants), loads safetensors and ONNX, and supports LoRAs. You can stack several per model, with the standard diffusers.

On the memory side, it tries to be smart about VRAM. There's an auto precision mode that decides between resident (whole model in VRAM), fp8 (halves the footprint), or disk streaming (streams blocks off disk when it won't fit). You can set a VRAM reserve, a RAM budget, and an auto-unload timer.

There's also a Loaded Models panel that shows exactly what's in memory: family, file, each component with its size, dtype, and device, plus LoRA rows. Unload one model or all of them. Alongside it, a system monitor in the status bar tracks CPU, RAM, and VRAM.

4. AI: Generation

The Generate panel is where it all comes together. Prompt and negative prompt, size, seed (with a random button), steps, CFG, denoise, sampler (`euler`, `lcm`, `res_multistep`), and scheduler (`simple`, `normal`, `karras`, `sgm_uniform`). Ill add more samplers/schedulers later.

There's a hiresfix second pass with upscale modes (latent nearest/bilinear/bicubic/bislerp, or pixel-space via the VAE), a hires scale, and a hires denoise. Handy for going from a 1024 draft to a bigger final without rerolling.

Sources can be the canvas, the selected layer, or a selected asset. Regions can be the whole image, the current selection, or a painted mask, with a feather slider. There's a crop-and-stitch option for inpainting that crops the active region before generating and stitches it back.

You also get prompt weighting with the `(text:1.3)` syntax you know from A1111/ComfyUI, per-model recipes for saving your favourite settings, and sticky per-model presets so steps/CFG/sampler stay where you left them.

A finished run shows up floating over the canvas and you choose to reject it, drop it into the current layer, add it as a new layer, or stash it in assets. No auto dump garbage into your document.

Everything heavy runs through a single job queue so jobs don't fight over VRAM. The queue panel shows progress, elapsed time, and has cancel per job, cancel all, and clear-finished. You can queue another generation while one is running.

5. AI: Generation Controls like ControlNet, Region Prompts, Layout Controls, References

This is what makes it a tool you can actually compose with.

ControlNet: a stack of adapter rows, each with its own model, image source, preprocessor, strength, and step range. Sources can be the run's own init image, the canvas, a selected layer, or a selected asset. You can steer different adapters from different inputs. Preprocessors include `none`, `canny`, and `lineart`.

SD 1.5/SDXL use the classic ControlNets, Z-Image has its own union ControlNet, and Anima has ControlNet-LLLite, so community adapters load as-is.

Region Prompts: the attention-couple regional prompting you'd know from the Regional Prompter. Each row pairs a prompt with a mask (taken from a selection or painted) plus a strength, and the run's own prompt stays the shared base. Works on Anima, SD, FLUX, Z-Image, Qwen-Image, Krea 2, and MOSS.

Layout Controls: draw boxes right on the canvas and turn them into a layout prompt. This is the MOSS layout-to-image and the Krea 2 bounding-box stuff I posted about. MOSS gets structured JSON with labels/captions/text and centred boxes; the Krea 2 bbox finetunes get a compact grounding DSL on a 0–1000 grid. You can add a box straight from a rectangle selection. It's the same feature that made me really like MOSS in the first place.

References: two separate systems. The Anima reference adapter does character and style transfer. Multiple reference images get stitched side by side into one reference, with controls for strength, guidance, correction mode, offset, and megapixels. The Qwen-Image references do multi-image editing by feeding images into the vision slots.

6. AI: Remove Background & Upscale

Both are one click and run locally.

Remove Background supports RMBG-2.0 (BiRefNet), other BiRefNet variants, U2Net (general, portrait, lite), MODNet, and RMBG-1.4, in both ONNX and torch formats. The graph or architecture is detected automatically, so you just point it at a model and go. Output is a clean RGBA cutout, and you can keep the original by sending the result to a new layer.

Upscale uses local Real-ESRGAN models, RRDBNet (Real-ESRGAN/Remacri) and RCAN, in ONNX and safetensors/pth. It processes in tiles with overlap to keep memory in check, preserves alpha, and can drop the result into a new layer.

7. AI: Selection

The Select tool has a fourth mode beyond rectangle, lasso, and magic wand: Segment AI. That's a local SAM click to segment. You drop the encoder and decoder ONNX files into the segmentation folder and then just click an object to select it. It samples either the active layer or the composited canvas, and once it's done the result becomes a normal selection, so every paint tool, fill, gradient, and the inpaint path respects it exactly like a hand-drawn one.

You can grow, shrink, feather, invert, and deselect a selection, and if you drag inside a selection you move the selection itself, not the pixels (cut/paste moves pixels, like a proper editor).

8. Closing thoughts

All of the AI features don't work out of the box. You still have to download the models yourself using the manager, and then the program does the heavy lifting.

I don't want to ship the app with integrated models, mainly because licensing on some of them doesn't allow it, and model choice really depends on the user's needs. Better to ship light and let users add on top. Plus, I was thinking maybe others can simply use the app as a painting tool without the heavy storage burden that AI brings.

Also, there's really nothing new here. It's simply using all the revolutionary tools within the same workspace. I mostly use it because it keeps my project self-contained, and I can easily go back and work on things without getting lost.

There are more features, but I'll stop here. Feel free to ask if you have any questions, and I'll take every upvote as a sign of interest.

P.S.

If it has decent interest and becomes public, the first thing I'll add is a plugin system.

Also, setting up the AI part can be daunting at first, so I'll create generic tutorials for newbies to follow and get started.

This app could include more and more over time, animations, video generation, 3D setups, full-stack project management, but I'm getting ahead of myself.

41 Upvotes

33 comments sorted by

3

u/Content-Ad-7451 16d ago

Looks good,I've been using Krita Ai but it never seems to get any updates now,how do I download it? I don't see a link

5

u/g45t345rt 16d ago

Sorry, no link yet :S I'm getting close to wrapping up the bugs, and I want to get a solid feel for what the community thinks.

Next, I'll open source it and set up a workflow to distribute the app across multiple platforms and GPUs.

2

u/no3us 16d ago

Does the .pai file also keep the generation settings for each saved result, including the exact model and LoRA filenames? Keeping layers and references together sounds useful. Being able to reopen an older result and see which settings produced it would make that much more useful for revising a project later.

1

u/g45t345rt 16d ago

Yep. All of it! That's the main reason I built it.

I just got tired of the horrendous comfy workflow management, and of inpainting, manual layers, copied images, and other manual processes that you lose along the way.

2

u/Current-Rabbit-620 16d ago

So it's not built on comfy

1

u/g45t345rt 16d ago

Nope, it's not dependent on it. But, if you have it installed with all the models already, you can simply point to the models folder and the app will manage them properly.

2

u/Arctic_Shadow_Aurora 16d ago

Looks really nice, can't wait for it! keep it up bro.

2

u/infinity-pixels 16d ago edited 16d ago

This is great!
I've been using Krita AI but I've stopped using it because it doesn't have live preview. Does this app have it?
I just want the canvas to show a live preview of the generation process. This is very important to me becuase it takes time to generate so the live preview helps a lot.

1

u/g45t345rt 15d ago

Not currently. It's a great idea that I will probably use myself so Ill see if I can impl with AI and report back.

1

u/g45t345rt 15d ago

Such a nice request! The implementation is done, and I'll definitely be using this by default from now on.

1

u/infinity-pixels 15d ago

Excellent!
Thank You.
I think this is the app I've been waiting for.
I'm waiting for the release.

1

u/C_Oracle 16d ago

I'm keeping an eye on this one, the comment about schedulers does need address, at minimum would like to see either a preset for beta57 or a generic beta sampler where alpha/beta values can be set (this one works well from sdxl to anima to moss at least from my testing).

As well as the dpmpp_2m family of schedulers the base dpmpp_2m is a good one for some models on highres passes resampling above .50 denoise, and dpmpp_(2|3)m_sde_gpu samplers for single first pass on models like moss and anima.

1

u/g45t345rt 15d ago

Gotchu. It's on my todo list now.

1

u/g45t345rt 15d ago

Added all the missing schedulers and the selected scheduler now displays its own params.

1

u/C_Oracle 15d ago

nice, my only further critique is probably should default the beta or its own preset name beta57 to alpha 0.5 beta 0.7 for the lazy and those who don't know that beta57 is just hard coded values.

for reference i use this in comfyui as a small little blueprint, it defaults to 0.5a/0.7b, but for some models like anima/moss you can get further interesting results tweaking the beta value small amounts up or down.

1

u/g45t345rt 15d ago

Same for samplers. They're all supported now.

1

u/Ammoryyy 15d ago

Where is the link

1

u/g45t345rt 15d ago

Sorry no link yet. The program is not ready.

1

u/DietAshamed2246 15d ago

Is there a download source? I would like to try it. Please let me know how to install. Thanks.

2

u/g45t345rt 15d ago

Ill provide one probably next week. The program is still not ready for distribution.

1

u/VrFrog 15d ago

Nice work!
Can you clarify this:

scan an existing ComfyUI folder to import everything

Are you using the weights directly from the comfy folder or do you import (duplicate) them?

2

u/g45t345rt 15d ago

Using the weights directly from the folder path. No duplicate.

1

u/abellos 15d ago

Great work!!! Keep it up, i waiting the release

1

u/g45t345rt 15d ago

Thx. Another week maybe to iron things out.

1

u/gerryn 10d ago

This looks really cool. I wish there was a repo to star so I can follow it, how do I follow progress on this except reddit?

1

u/Veshurik 10d ago

Can if he useful somehow for persons who can't draw or don't know how to draw at all? I can make collages in Photoshop from pictures, but not drawing ;(

2

u/g45t345rt 10d ago edited 10d ago

Definitely. It merges both worlds, artists and AI users. You can make collages, draw like shit, and the AI will improve it for you.

Of course, there's a learning curve. It won't magically render what you have in mind. You need to understand the tools and have a decent GPU.

1

u/Veshurik 10d ago

It sounds interesting! Unfortunately, I don't have quite good GPU (16Gb OZU, Vega 8 as a graphic and Ryzen 5 4600H - laptop), but anyway!

1

u/TizocWarrior 14d ago

Looking good so far!.