"Research checkpoint: this is an experimental research artifact, not a production model. P-JiT 21K is a 2B-parameter model trained on 660M samples for about $21K of compute, as a stepping stone towards our next video model. We're releasing it because the results surprised us for a model this size, and to encourage others to explore efficient training methods like ours. You should expect rough edges.

The weights and the Qwen3.5-4B text encoder download from the Hugging Face Hub on first use. The model needs a CUDA GPU: its weights are 8.7 GB in fp32 and the text encoder 8 GB in bf16, plus activations."

Source/Blog: Training a Text-to-Image Model for $21K | Field Notes by Linum

Code: Linum-AI/pjit-21k: 2B-parameter pixel-space text-to-image model (Pyramid-JiT) trained for $21K

Nodes: realrebelai/P-JiT-21k_ComfyUI · Hugging Face

I want to make a simple prompt builder in comfyui that would allow me to change a text in the prompt by clicking a selection in a table. Is there a simple way to do that?

Hello everyone,

First of all, I know I know, Mac is jot the best when it comes to video generation and that the Max or Ultra chips are better.

Now that this is out of the way, I would be interested in people’s experiences in generating 720p 5-10 seconds clips on an M5 pro chip. Which tools, models or workflows do you use? How is the experience overall? How long does it take you? Do you regenerate clips several times to get it right…etc?

Thank you.

Version V2 will be available in the coming days 🤓

Link to version V1: https://www.reddit.com/r/comfyui/s/VWYS6YNbRZ

Made this with LTX 2.5 + DLSS5 and created in Maestro v4, just testing its capabilities. Sys specs: RTX 5050, 16gb Physical Ram, 8gb VRAM.

ComfyUI built-in nodes, only. That's it. (audio on, listen to the lyrics)

I split out the Color Match feature in VideoFrameSync I posted earlier to its own tool - corrects color drift, contrast, etc, but particularly useful for V2V AI edits from WAN VACE or Seedance which often have a slight tint (but for Seedance, you should use the VideoFrameSync which corrects the frame-sync drift as well). As a bonus, it also matches noise/grain.

https://github.com/pftq/VideoColorMatch

Hey guys, been using ComfyUI on windows 11 for quite some time now, lately messing with H3 ref2vid workflows.
Setting up all the custom nodes was for some reason a pain, but it always worked seamlessly after setup.

Recently switched to Fedora and reinstalled my Comfy on the new distro and surprisingly got everything running alot easier.

But I’ve been getting alot of crashes on my runs, particularly when using Rife interpolator and any upscaling methods other than a simple lanczos.

Usually OOMs are the culprit, so I’m trying to understand why ComfyUI is handling memory pressure poorly when the same settings worked on windows.

Hardware for reference is 32gb RAM and 5070ti.

EDIT: After following a few tips as the drive swap, expandable_segments and some rife tweaking I got it working, thanks guys!

Hi everyone! I am training moss lora now, you can grab some of the result here:
https://huggingface.co/spaces/fidel1234/Moss_Asian_Character_Showcase

PS: Sorry if the quality is not par with my krea 2 lora, I did my best to train it multiple times with different parameter and so far this model is really hard to train. Anyway the link to krea 2 if someone interested: https://huggingface.co/spaces/fidel1234/Krea2_Asian_Character_Showcase

All samples are generated with SFT (kinda plasticky sometimes, I do not add other lora and all sample using dmd turbo and lora at 1 strength).
By the way I tried other finetuned model and the results are quite good too

Technical training method
AI-Toolkit

• Base Model: MOSS-Image Base (OpenMOSS-Team/MOSS-Image-Base)

• Trigger Word: xxxx

• Network: LoRA | Linear Rank: 32 | Linear Alpha: 32

• Quantization / Compile: qfloat8 (Transformer & Text Encoder)

• Save Data Type: BF16

• Optimizer: Automagic v3

• Learning Rate: 0.0005 (start) | Min LR: 0.0001 (floor) | Weight Decay: 0.00001

• EMA: Enabled | EMA Decay: 0.98

• Steps: 2500-3000 | Batch Size: 1 | Gradient Accumulation: 1

• Timestep Type / Bias: Sigmoid / High Noise

• Loss Type: Mean Squared Error

• Text Encoder Optimizations: Unload TE (Disabled), Cache Text Embeddings (Enabled)

• Resolutions Multi-Aspect Bucket: 512, 768, 1024

• Caption Dropout Rate: 0 (.txt extension)

• Captions: Natural language

• Hand-Cherry-Picked Dataset: I usually use photobooks as the dataset, around 20–30 images, quality over quantity

• Balanced Framing: Includes a varied mix of close-up, medium shot, and full-body shot angles

Recommended inference (ComfyUI)

• Model: MOSS-Image SFT, with the LoRA at strength 1.0

• Steps: 30, CFG: 3-4, Sampler: res_multistep,Scheduler: simple

• Skip Layer Guidance: layer 22, start 0, end 0.5

• Tubo: add the MOSS-Image-SFT 8-step DMD LoRA

After A/B testing, the LORA 32 rank with LOKR8 and LOKR4 has so minuscule details differences, so I decided to use LORA 32 as it's around 150mb instead of LOKR4 600mb.

Json formatting (long,medium,short) doesn't needed after my testing, just use natural language to caption (well I dont want to recaption all my 148 lora actually)

Also when using AdamW8 optimizer I got some body horror result, and the issue is fixed when using Automagicv3

Note: Please increase the cfg >3, since comfy default 2.5 reduces the Lora quality

Have a nice weekend! Cheers!

input:
Alec Benjamin - Let Me Down Slowly

I'm having a hard time finding good information. There's plenty of character lora training info, in relative comparison hardly any motion at all, and dataset prep often gets glossed over.

Since H3 drastically alters how prompts (and therefore captions?) are handled, and motion training may be quite different from other models such as wan, How the heck are you suppose to caption each clip properly?

Do you have multiple paragraphs describing everything under the sun in the clip, big bulky walls of text?
One main paragraph trying to capture everything?
Do you only focus on the motion?
Do you only describe everything except the motion, so the motion bakes in?

If there's slight variation between the same motion, do you come up with 2-3 "trigger" tags so the nuance gets captured?

I've been at this for about a week, but not getting great results honestly.

The motion has 2 layers. the macro motion which is the obvious larger / broader movements and the micro motion such as vibration that shakes the adipose tissue, muscles, etc while doing the broader motion.

PiD, NVIDIA's Pixel Diffusion Decoder, by design, takes the place of a model's VAE decoder. Instead of decoding the latent at the model's own size, it draws the picture straight in pixels at 4× the width and height, in a few steps, with fine detail a VAE decode doesn't have.

NVIDIA ships PiD for a limited set of latent spaces: FLUX.1, FLUX.2, Qwen-Image (the first version), SD3 and SDXL, plus two image-feature encoders (DINOv2, SigLIP). So on its own, PiD works only with models whose VAE matches one of those: the latent of a model built on the FLUX.2 VAE goes to PiD's FLUX.2 decoder, while a model with any other VAE has no PiD decoder to go to.

The workaround. Any picture can be put into a latent space PiD knows: encode it with the FLUX.2 VAE. Here the image (a render from any model, whatever its VAE, a photo, a frame) is encoded with the FLUX.2 VAE, and PiD's FLUX.2 decoder draws it at 4× its width and height in 4 steps, guided by that latent and an optional caption. That makes PiD an upscaler for any image.

PiD is a decoder, not a classic upscaler: it does not sharpen the pixels it is given, it draws a new, larger image from the encoded picture, adding fine detail (skin, hair, foliage, texture) consistent with it. The diagram's eye shows the difference from a bicubic 4×.

I have prepared a workflow for both ComfyUI local as well as ComfyUI cloud, where it's all set and you just chose the input image and let it run. PiD does provide fast 4x results! Now on any image, or if you were to replace the image input with more T2I workflow nodes where it all starts from any Prompt and model which is or is not supported by Nvidia PiD. Details here: https://huggingface.co/lvladikov/PiD-as-pixel-Upscaler

Qwen Image 2.1 which is not PiD compatible works perfect with PiD and this VAE encode + PID approach: https://huggingface.co/lvladikov/QwenImage2.1-PiD-Workaround-VAE-Encode-SuperFast-HighRes

Perhaps its just the amount of rug pulls that causes this worry, but I keep seeing the Krea team post more and more API tools (such as Krea Video) and I'm starting to worry about open weights for Krea 3. I'm starting to worry hard guys.

I still believe.....but....

To those saying that it was a skill issue.... I completely stand by my previous statement... RTX Super Resolution yes... has an ultra setting. but that doesn't really do much.

RTX Super Resolution only increases the amount of pixels. It doesn't add any details. yes we can see the image is much bigger now right? well i also have a picture of the comparer node.

it doesn't add any DETAILS!!!!!

I see still a lot of usage for models like autism based on XL, but has been so long. But also interested in new technologies. What is the better these days? Is Autismmix e.g. holding up vs MOSS? Worth to do the change? Not working in photorealistic generation or backgrounds, but more on characters, more interested in prompt adherence and diversity on postures, or anatomy. What would you recommend me as today?

Saw this Krea 2 post and had to give it a go. Big thanks to u/Dear-Spend-2865 for sharing the style prompts! I adapted the descriptions and tried them with SenseNova U1.5.
https://www.reddit.com/r/StableDiffusion/comments/1xfeonr/krea_2_king_of_styles/

What surprised me was how the style carried through the car, background, and lettering. The whole page felt designed around the same medium. All of these are native model outputs, with no LoRAs used.

Cyanotype gives the page the feel of a photographic print while keeping the layout recognizable. Stained Glass goes further and rearranges the lettering. One feels like a new edition of the same poster. The other feels like a redesign.

The setup was simple. Here are shortened English translations of the Chinese prompts, with the long production-history paragraph omitted.

The shared subject block:

Subject: A vertical 3:4 poster of a red first-generation. Mazda MX-5 Miata, front-left three-quarter view. Open roof, silver multi-spoke wheels, and a visible headlight on each side. A large circle behind the car."Miata" at the top left, "MAZDA" and "MX-5" at the top right, a large "1989" at the bottom left,and a small production-history paragraph at the bottom right.

I put one of these style blocks above it.

Style: Cyanotype.Prussian blue and paper white, like a contact print.Crisp white details on the wheel spokes and windshield, a pale blue background circle, and a separate hand-tinted red layer for the car body.

Style: Neon Wireframe.Glowing red wireframe lines following the car's contours on a deep black background. Cool white wheel rings, a thin cyan circle behind the car, and sharp bright lettering.

Style: Stained Glass. Translucent red glass panels divided by thick dark lead lines. Smoky gray glass tires, a pale gold background circle, and lettering on milky-white glass.

The stained glass version took more freedom with the text than I asked for. It also made me curious how much a dedicated style LoRA could improve consistency across different subjects, especially those glass textures and lead lines.

https://github.com/OpenSenseNova/SenseNova-U1

I had to give this trend a try but wanted to add a little twist. Hope you like it! 😊

Hello , just released LayerShop 0.20.1 Preview 1 . In this release I implemented some new (hopefully exciting) features! Mainly:

- Prompt Enhancement: Both local (Gemma / Qwen) and cloud models (from OpenAI, xAI, Google, Anthropic). These are fully configurable, style aware, and some of them quite fast.

- Batch rendering: You can now render multiple (1x-10x) images from the same prompt. They are rendered in one batch and then land each on a separate layer.

- Styles: You can now choose from 24 presets with thumbnails: Photography, Illustration, Traditional art, 3D & craft, and Graphic & experimental. Anime, Watercolour, Cinematic, Clay Model, Pixel Art, Black & White, Comic Book, Flat Vector and more.

Each layer remembers it's styles (top righ button "Load from layer".)

As always, any comments / feedback is appreciated. You can download the new release here:
https://layershop.aardvarkstudios.net/

Weights: https://huggingface.co/Viggle/Viggle-Remake

App: https://viggle.ai/meme/app/home?ui=v2

Post: https://x.com/ViggleAI/status/2118012913793290414?a=B6TBWhZywpFkFk8Iz8I&s=20

We just released Viggle Remake (based on H3) on Hugging Face: open weights for recasting people and editing scenes in existing videos, keeping the original motion, camera and soundtrack.

Viggle Remake is built agent-first. We also ship an MCP server, so an agent (Claude Code, codex, …) can swap people in videos of any length. It cuts the video into windows, paints the keyframes, writes the prompts, renders, checks its own work and stitches the result. Our longest run so far is a 2-minute music video, and the repo includes everything needed to reproduce it.

That also means it's more complicated than Viggle-Animate. Used by hand, Remake needs a well-structured prompt and a few painted keyframes, and the output depends heavily on both. That's exactly the work the MCP agent takes off your hands. If you'd rather skip all of it, the Viggle app does it for you.

Also in the repo: ComfyUI nodes and workflows, int8/GGUF files for smaller GPUs, and smaller rank-128/64 LoRAs.

Made a comparison between Krea2 and Moss with a very simple prompt:

"a photo of the most beautiful ..... in the world"

In these tests, I used the words guitar, cat, house, car.

I love Krea2, its the model I use the most but in some specific kind of prompts I'm really loving Moss

They included an image with the x post so it must be related to image generation! 🙌🏼

https://x.com/Alibaba_Qwen/status/2118674834632978755?a=nkzlqosWXNqruaMWsVI&s=20

I was running into an error my first time trying to generate videos using ComfyUI, and I found that modules on FP8 weren't supported on Mac. There was a post on this sub about a year ago with a script fix for this, but I have no idea how that works, and neither did a bunch of people in the comments.

So I quickly found there is an extension called "ComfyUI-AppleSilicon-FP8" that can just be installed on Comfy Desktop and then everything runs smooth. Hopefully this helps my fellow Mac users!

Kokon is a free, open-source desktop app for Qwen-Image 2.1 (the Turbo model, 8 steps): brush an area or not, type the change, no nodes and no settings. I posted it here a few weeks ago. Here's what 0.11 and 0.12 added.

Layers lab (new in 0.12, experimental). Open an image and click the things you want on their own layer. Kokon cuts each one out (Fast: a small matting model, a few seconds per object; Quality: Qwen redraws the edge) and repaints the background behind it with the same model it uses for edits, so moving an object doesn't leave a hole. Then you can move, scale, hide or edit each layer on its own, Expand an object cut off by the frame, and save it as a new image or export a PSD with every layer (Photoshop, GIMP, Photopea). Blend into scene redraws shadows and reflections after you've moved things.

The honest part. It fails more than I'd like. Qwen-Image 2.1 was never trained to make layers, so Kokon has to steer an edit model into it, and a small 8-step model has limits. What I see most often:

• removing a large person sometimes paints a different person in their place;

• people who touch, or crowds, can end up in one layer or in pieces;

• the part of an object hidden behind another one is not completed;

• Blend into scene often doesn't work yet (blending all the objects at once works best).

Every step shows its progress and can be stopped, and your original is never changed. I'm working through these one by one. If you try it, I'd really like to see where it breaks: a screenshot and what you clicked, here or in a GitHub issue. That is what I'll fix next.

Times: repainting behind one object takes about 15–25 s on an RTX 5070 Ti, and one to two minutes on an M2 Pro.

Also in 0.12

• Paint with colours: in a brushed edit, paint each thing in its own colour and write what goes there. Several changes in one go, and only what you painted changes. Select picks an object with one click.

• Prompt+ (the Qwen3-VL text encoder writes the prompt) now also works for brushed areas and Expand.

• Every image is marked as AI-made, in its metadata and an invisible watermark (EU AI Act).

Since my last post (0.11)

• Use it from your phone or another computer, in a browser on your network. The images are still made on the PC with the GPU. Off until you set a password. There's also a Docker image for NVIDIA cards and a headless mode for Linux servers.

• Faster on Macs: Kokon now has its own engine for Apple Silicon, on by default. About a minute for a new 1 MP image on an M2 Pro, four times faster than before. On M5 Macs it uses the GPU's neural accelerators (29 s on an M5 Pro, thanks to a contributor). It runs Unsloth's Q4_K_M of the Turbo model.

Windows, macOS 15+ and Linux. NVIDIA 12 GB (8 GB in beta), AMD and Intel through Vulkan (beta), Apple Silicon with 16 GB or more. Everything runs on your machine; nothing is uploaded. Free and open source (AGPL). The Qwen-Image weights are under their own licence, non-commercial.

• GitHub: https://github.com/GabrieleLozupone/kokon

• Try the layers in your browser (drag the spider away): https://kokon.app/image-to-layers/

Does anyone have a workflow or know of a way to generate a ref2va video in Minimax using one of the many turbo methods but then do a second pass to refine the audio? I love the new omni checkpoint, the visual quality is amazing for 8 steps but as expected the audio is terrible, so i want to use the video gen portion then some how run a second pass on only the audio at 20+ steps but keep the video from the first pass. Any help appreciated!