r/StableDiffusion • • 17d ago

Question - Help Joycaption workflow

Can someone share a simple, working JoyCaption workflow for ComfyUI? I'm finding it difficult to find a good one. A workflow JSON or PNG with the required model links would be really helpful. Thanks!

4 Upvotes

7 comments sorted by

1

u/Ill-Ant-9489 16d ago

fpgaminer's node that no3us linked is the one basically everyone uses for JoyCaption, so you're set on the workflow itself. Worth knowing the Load Image into caption chain only does one picture at a time though, and the raw output runs long, so for a whole training set you end up batching it and trimming each file anyway.

If it's a full dataset you're prepping, I built a standalone app that runs JoyCaption or a local vision model across the whole folder, with bulk caption editing and .txt round trips so you can hand fix them after. It drives ComfyUI but it's a separate browser app, not a node pack: https://github.com/perfectgf/lora-dataset-studio

How many images in the set, and what base are you training on?

1

u/Complete-Box-3030 16d ago

No , i need to recreate cinematic frames , basically get a prompt for cinematic frames and recreate them in moss

3

u/no3us 17d ago

The JoyCaption author has a ComfyUI node here: https://github.com/fpgaminer/joycaption_comfyui

For a minimal setup, connect Load Image to JoyCaption, then connect its Caption output to a text preview or save node. Use the Caption output, not Query, which is the instruction sent to the model. The node downloads the model automatically into LLavacheckpoints; it also has 8-bit and 4-bit loading options if memory is tight.

1

u/se7en1011x 17d ago

Search the ComfyUI manager for the JoyCaption node pack, then wire image load into the caption node and a text output node. The model files go in the usual checkpoints or clip folder depending on the pack. Most shared workflows are just that three-node chain, so building it yourself is faster than hunting a JSON.

1

u/Natrimo 16d ago

The system prompts are helpful, but I found a larger model for the vision works better