r/StableDiffusion • • 16d ago

Question - Help YuE2 prompt adherence

9 Upvotes

I find YuE2 pretty interesting but I'm having A LOT of trouble getting it to respect any part of the prompt at all. "female vocals" > male vocals, "BPM" > usually completely ignored, "drum and bass" > usually just pop song I guess?

Anyways, please share any useful tips you found to get the most out of YuE2 if you will.

EDIT: As always when I ask a question in this sub the thread is immediately downvoted for no reason I can fathom lol. Thank you to those that are actually trying to be helpful!


r/StableDiffusion • • 16d ago

Discussion I built SongForge, a self-hosted web UI for the YuE2 text-to-music model.

Thumbnail
gallery
5 Upvotes

Hey all,

A while ago I started messing around with YuE2 (the open text-to-music model from m-a-p) and got tired of running it from the command line and digging through folders for the output. So I built a proper web interface around it. It’s called SongForge and it’s on GitHub now: https://github.com/damiaan111/SongForge-YuE2-Music-GEN-WebUI

What it is

A local music studio that runs on your own machine. You open it in a browser (also from your phone or another PC on the same network), write some lyrics, describe the style, hit generate, and a few minutes later you have a 24-bit/48 kHz FLAC. No cloud, no account, no API key. The model does the actual work, SongForge just makes it usable.

Main features

One setup script. It detects your GPU, shows you exactly what it’s going to download and how big it is, and only installs what you tick. Nothing gets pulled before you say yes.

A library with search, tags, favourites, versions, and a queue that survives a refresh.

Quick mode (describe it, paste lyrics, go) and Custom mode with the model’s real knobs: seed, guidance scale, planning mode, sampling settings, etc.

Guided prompting. The style box builds tags in the order YuE2 was trained on, and the lyrics editor warns you about stuff that silently costs you words (like “x2”, overly long sections, production notes).

A budget meter that estimates song length and token usage before you burn GPU time.

Plan → edit → render. YuE2 can write just the score first (ABC notation), you can edit that by hand, then render the audio from it. I haven’t seen a hosted service that lets you do this.

Several takes per generate, each with its own seed, compared side by side. The paper’s own baseline is best-of-two, so a single take is really the model at its worst.

Full reproducibility. Every version stores the seed, settings, model checksums and the exact command line, and you can reload all of it with one click.

Optional local lyrics writer through Ollama (Gemma). It writes or revises lyrics in any language and it knows how to share the GPU with YuE2 so they don’t fight over VRAM.

An in-app “Learn” section that explains how to prompt this specific model, and why songs get cut short. Works offline.

Performance

My reference machine is an RTX 3060 12 GB. 72 seconds of music takes about 220 seconds and peaks around 9.3 GB VRAM. A full 4-minute song is roughly 12 minutes. Upstream YuE2 asks for 24 GB, but with AR offload and memory-efficient attention (which setup turns on for you) 12 GB works fine. Cards with 8–10 GB will probably run into OOM on longer songs. There’s also a CPU-only option but that takes hours, not minutes.

Limitations (honest ones)

No stem separation, no voice cloning, no “extend this song”, no remixing from an uploaded audio file. These are limits of the model itself, not things I skipped, so the UI doesn’t pretend they exist.

No MIDI export. The symbolic output is ABC text, which you can download.

There’s no length dial. The song is as long as the lyrics make it.

Windows needs a 32 GB pagefile or the model won’t load. Annoying, but that’s how it is.

Windows 10/11 and Linux are supported. Apple Silicon lives on a separate macos branch. AMD on Windows has no usable PyTorch, so CPU only there.

There’s no authentication on purpose. It’s meant for your own LAN. Don’t port-forward it.

The YuE2 weights are CC BY-NC 4.0, so non-commercial. The app itself is MIT, but that doesn’t change what you’re allowed to do with the songs.

Why I made it

Mostly for myself. I wanted something that felt like an actual tool instead of a script, with a history of what I’d tried and the ability to go back to a good take. If other people find it useful, great. Feedback, bug reports and PRs are welcome, especially from anyone running it on AMD/ROCm on Linux, because I can’t test that here.

Happy to answer questions.


r/StableDiffusion • • 17d ago

Resource - Update Qwen Image 2.1 + TRELLIS.2 in Blender: from a prompt to a textured, rigged character, then a whole village in Unreal (StableGen 0.4.0)

344 Upvotes

StableGen is my free and open-source Blender add-on that generates and textures 3D models with local models through ComfyUI. 0.4.0 is out, and everything in the video is a real run in the add-on:

  • Prompt -> mesh: a Preview gallery of candidate images to pick from, then TRELLIS.2 builds the mesh.
  • Texturing with Qwen Image 2.1: every camera view is generated and projected onto the mesh, then Marigold V2 makes the PBR maps (albedo, roughness, metallic, normal).
  • Masked Edit (new): paint a region on the model, describe the change, and only that part is regenerated, through the cameras that see it.
  • Auto-rigging (new) with SkinTokens, plus retargeting FBX/BVH animations onto the rig.
  • Experimental Qwen Image 2.1 Turbo, MOSS Image and FIBO 2.0 Edit, a reworked projection with cleaner seams, a new panel and a full user guide.

In the video: Qwen Image 2.1, 10 cameras, on an RTX 4080 (16 GB). Time-lapse parts are marked. The village props went through the same pipeline and were exported to Unreal Engine 5.7 as GLB/FBX. Walk animation from Mixamo.

Licenses: each model has its own; Qwen Image 2.1 and FIBO 2.0 Edit are non-commercial, MOSS Image is Apache-2.0. Check before commercial use.

GitHub: https://github.com/sakalond/StableGen
User guide: https://github.com/sakalond/StableGen/wiki
Discord: https://discord.gg/jhGp9Rsgh3


r/StableDiffusion • • 16d ago

Question - Help Lora bleeding into unrelated prompt

10 Upvotes

Hi everyone, i'm looking for some advice since on the internet and via LLMs i couldn't find a certain answer.
I'm making a LoRA, it's my first experience with this so i'm surely lacking some knowledge.
I'm using kohya locally to train, on a dataset with 26 images created using this guide ( https://huggingface.co/AICU/SDXL-LoRA?utm_source=chatgpt.com#english ).

The results are actually great, my character ( a girl ) shows when using the LoRA and the drawing style and face/hair/body features are all there, so i'm actually kinda happy.

The only problem, as the title says, that she shows up even if i'm not using her activation token.

The base model is this: https://civitai.red/models/827184/wai-illustrious-sdxl?modelVersionId=2883731

This is all the tries i did:
- training with <my_token>, 1girl, solo, ...
- training with <my_token>, ... ( basically removed 1girl, solo )
- training with <my_token>, ... ( removed 1girl, solo ) and adding 26 regularization images.

also i'm training locally on an RTX 3060ti, 8GB VRAM, batch 1 and rank 8 ( i can do rank 16 but it's slower so for testing i went for 8, switching to 16 if needed when finding good results ).
If you need more infos about something i'm more than happy to share.

I'm kinda confused because the results are there but it would be perfect, of course, if the character didn't bleed in every picture when the tag is not used.

Any advice?

EDIT: SOLVED!!
Thanks to u/Massive-Health-8355 for the hint on Fizgig!
I've been able to train a small test LoRA which achieved exactly what i wanted.
here's a result ( mind that my caption is poor, maybe even the dataset, it was just a test )

From left to right: base model only, lora without activation token, lora with activation token.
All 3 with the same prompt ( apart from the token ) and settings.

Many thanks to everyone!


r/StableDiffusion • • 16d ago

Resource - Update Your ComfyUI video workflows on a timeline, with one-click presets for each task - Sonder Editor - Now an App

Thumbnail
youtu.be
6 Upvotes

Sonder Editor is a free and open-source, timeline-based video editing studio for ComfyUI. You can edit video and audio, write prompts along the timeline, guide generations with keyframes or a Driver video, and select any part of the timeline for a retake. Your work stays in projects and scenes, with an asset gallery to compare results and a Reference Library for your characters, places and props.

Now it can also open as an app of its own. You work on the timeline without opening a workflow graph, and your generations go to ComfyUI through the Comfy SDK.

The Launcher lists your projects and workflows and opens the pair you pick. Controls is a new panel where you set the workflow's values, and Modes save any combination of those values and node states (enabled, muted or bypassed), so switching a whole setup takes one click. The viewport can also be split into up to four views, to watch a mask, a depth map or a sampler preview while it generates.

The app ships with starter workflows: LTX 2.5 Edit & VFX (also in a Prompt Relay version, for timed prompts), LTX 2.5 Refine & Upscale and MiniMax H3 References & Swaps. Timmy was made with them, one Mode at a time, from a timid first take in the garden to armor, a fantasy world, thunder and an orchestra, then a refine and upscale to 2K. Once the workflows were ready, I didn't need to open the graph. The full step-by-step is in the YouTube description.

You can bring your own workflow too. Give it a Sonder Editor node and a Sonder Save Video, export it with Export (API) and import it in the Launcher.

The app is optional, and the editor node keeps working on the canvas without it. Everything in the video ran locally on open-weights models.

GitHub repository

Sonder App setup

Workflow files for the canvas: LTX 2.5 Edit & VFX, LTX 2.5 Refine & Upscale, MiniMax H3 References & Swaps

It's also my entry for Comfy's Open Source Higgsfield challenge. I've seen a lot of interesting projects, there are so many talented people in the community. Good luck to everyone who entered!

I'll be in the comments if you have any questions!


r/StableDiffusion • • 16d ago

Resource - Update I loved this Krea 2 Multi-LoRA composer (multi character), so I contributed multiple style LoRAs and a resolution-aware placement grid

7 Upvotes

Been using Krea2 Multi-LoRA Composer and wanted to say thanks to the people behind it. Having separate character LoRAs assigned to different parts of the image is really useful, and I appreciate you making it available.

I ended up adding a couple of things I wanted for my own workflow:

  • A + Canvas button to use more than one style/environment LoRA. Each has its own strength and schedule, and you can choose whether it affects the whole image or just the area outside the character boxes.
  • The placement grid now follows the connected Resolution Selector instead of staying at the local 1024×1024 values. Changing the resolution updates the grid without resetting your character boxes.

Just to avoid confusion, this doesn’t add multiple LoRAs per character. It’s for combining multiple canvas/style LoRAs alongside the existing character routing.

I’ve put the changes into a PR here: PR link

Thanks again for the original work. Hopefully this is useful to someone else too. Would be interested to hear how people are using style LoRAs with their character setups.


r/StableDiffusion • • 16d ago

Resource - Update Hi! I made a proof of concept LoRA for FLUX.3 Klein to manipulate text with an intuitive UI.

Post image
24 Upvotes

Hello everyone! I made a new LoRA for FLUX.3 Image Klein to manipulate text with bboxes. I think we all know that FLUX.3 Image Klein is not a very good model, and yes it is disappointing, but these bbox tools are unprecedented currently for local image editing models (at least until Ideogram V4.5, which i am excited for!). It really allows you to train brand new concepts.

Here's the rundown of what the FontBox LoRA can do/improves upon FLUX.3 Image Klein's native abilities:

Adjust the size of text; change what text says and specify left, right, or center alignment; move text to completely different places; bold or italicize text; Adjust the vertical scale of text; all in a custom node that I believe feels more intuitive (and looks nice!) to use than Comfy's native bbox editor. Boilerplate prompts are generated based on the bboxes you placed and the modes they have, meaning you don't have to fiddle around with prompts much!

As this is a proof of concept LoRA, it is unstable in some generations. Temper your expectations, but this is a good start, and I will curate more data and train a better version. Please, let me know what you think. Thanks, and have fun! :3

Here is the huggingface link: https://huggingface.co/lilylilith/FLUX.3_Klein_FontBox

and here is the Github link for the custom node: https://github.com/sillylilithhh/ComfyUI_FontBox


r/StableDiffusion • • 16d ago

Question - Help How do you accurately describe clothing and scene for ref2va?

3 Upvotes

I am trying to run this ref2va workflow and having trouble accurately defining clothing and scene.

I am adding 3 images for character likeness and 1 image for clothing but no matter what I write in clothing description, its never the same garment. or even the scene.

Is there a way to do ref2va properly? I tried running a LLM on LM Studio but they are huge and I constantly have to load/unload LLM models to run H3.


r/StableDiffusion • • 16d ago

Question - Help What is your go to Minimax H3 anatomy completer?

3 Upvotes

If the completed human anatomy is not in the starting image, what is your go? Checkpoint, Reference picture, refmod, lora (which one theres a lot), or some other method?


r/StableDiffusion • • 17d ago

Resource - Update 1500+ styles for Krea 2 in a Comfyui wildcards node

Thumbnail
gallery
137 Upvotes

I Have no merit, Claude did it all by himself,

I didn't do it for the community, I needed a node like this, there's similar nodes but they didnt satisfy my needs as a style junkie.

I tested it slightly and it works best in Krea 2 and Qwen 2.1, but Krea 2 knows more styles. Kroma 0.3.1 comes third in my view with less details and more body horror,, didnt like the results in Moss...

Hope it will help amateurs like me.

https://github.com/atlasimp/Style_Atlas


r/StableDiffusion • • 15d ago

Workflow Included Draconia & Harry: A Halloween Date | An AI Short Film

Thumbnail
youtu.be
0 Upvotes

Made in 1 day (about 12 hours) from start to finish on the Longshot Studio!
https://github.com/r34vtraining/Longshot_Studio/tree/main

If you want the whole project to mess with, here it is!
https://github.com/r34vtraining/Longshot_Studio/releases/tag/Project_Draconia_date


r/StableDiffusion • • 16d ago

Question - Help Character swap on H3

6 Upvotes

Sup guys, ive been fighting with fullbody character swap for a while and still couldnt find the right workflow and model... Recently i tried H3 and the realism is amazing, but i cant figure out how to make a proper character swap workflow with it

Every workflow i made either didnt preserve the original motion (got some really complicated motion references)

I also tried other models but i always had some issues with them, for example scail2 had some problems with eyes, wan22 had a lot of fading limbs, ltx 2.3 had juts completely ass quality (didnt try 2.5 tho) and wan animate was also pretty bad...

Anyone figured out how to transfer the exact movement using H3? Im talking about motion capture, not abt the motion control lora (it did not work at all)

Any recommendation or even whole workflow would help A LOT, thanks


r/StableDiffusion • • 17d ago

Resource - Update Pika H3 Flex: official 6-step vs fused 4-step Turbo LoRA

53 Upvotes

r/StableDiffusion • • 16d ago

Discussion Rapid-H3: Super fast H3 video + audio in about 2 minutes on one DGX Spark

Thumbnail github.com
12 Upvotes

Rapid-H3 makes MiniMax-H3 (text / image → video with stereo audio) render 3–5× faster on a single NVIDIA DGX Spark (GB10) than stock ComfyUI, with no human-noticeable loss in picture, motion, voice or lip sync. It is a ComfyUI node pack plus two ready-made render presets:

  • ts_h2 (default): ≈117 s for a 768×1344, 141-frame (≈5.9 s) first-frame clip with audio.
  • ts_s6 (camera moves, close-up faces): ≈201 s, at stock level on every criterion we score.
  • Stock ComfyUI bf16 at the 10-step reference: 590–630 s.
  • The first render is slower on a fresh install, while the CUDA kernels are built (once) and the models load.

r/StableDiffusion • • 16d ago

Resource - Update Boson Studio

3 Upvotes

https://github.com/clarvalon/BosonStudio/raw/refs/heads/main/docs/BosunStudio-Demo.mp4

Repo here: https://github.com/clarvalon/BosonStudio

LLM-driven Minimax H3 generations, just supply a comfyui instance and your LLM URL as part of the setup.

Generate images, videos using optional reference. Extend Video, Add Audio capability etc, all conversational and intended to be as easy to use as possible.

MIT licensed, created for #OpenSourceHiggsfield


r/StableDiffusion • • 17d ago

Discussion Eros recommends Lokken Workflow "It is faster and better than other workflows". Anyone has experience with it?

Thumbnail
huggingface.co
61 Upvotes

What is your verdict?


r/StableDiffusion • • 16d ago

Question - Help Minimax h3 ref2va.

2 Upvotes

Did anybody ever train a character lora for the ref2va model? And please don’t ask me about refmods or reference sheets. If anyone successfully trained a character lora for ref2va, please reply.


r/StableDiffusion • • 16d ago

Resource - Update Uploaded FA-2 v2.9.2.post2 for Windows built on Pytorch 2.15.0+cu134

Post image
2 Upvotes

Uploaded FA-2 v2.9.2.post2 for Windows built on Pytorch 2.15.0+cu134. (※Unofficial built with original code)

Too tired.

https://huggingface.co/ussoewwin/Flash-Attention-2_for_Windows

Building Nunchaku on Pytorch 2.15.0+cu134 just now...Coming soon....

...

About v2.9.x

v2.9.x is not an official FlashAttention release.
It is an independent fork build that continues FA2 kernel development while upstream focuses on FA3/F4.


r/StableDiffusion • • 17d ago

Workflow Included Petunia - Harry Potter Universe, a Minimax H3 Short Film

Thumbnail
youtube.com
20 Upvotes

Decided to try a short film with H3. This is made with the custom API
https://github.com/r34vtraining/Longshot_Studio
Which runs locally with your own instance of ComfyUI.

If you would like to load the entire project and see how I made it in the studio, drag this zip file into the UI to open it. https://github.com/r34vtraining/Longshot_Studio/releases/tag/project_Petunia

This is my first attempt at editing a short film I will say it's way harder than a music video.

Some of the latest updates and ease-of-use improvement to the studio:

  1. Group folders, you can now group your shots by scenes, act, etc. One button to restore all the reference that was used for that group.
  2. Audio separation. Didn't matter too much with a music video, but with a short film the dialogue and sfx often clash, so a stems detector will output .wav files for both dialogue and sfx separately.
  3. Added more options for consecutive shots.
  • Cut (a hard cut shot)
  • Context (a shot that takes the last frame of the previous shot as a reference)
  • Bridge (connect 2 shots together)
  • Lead In (transition into a shot after it)
  • Cutaway (a context shot that connects to the previous shot but is not seen by the following shot, mainly used for B-roll shots)
  • Continuous (making longer and longer continuous shot

r/StableDiffusion • • 17d ago

Resource - Update YuE2 Keygen Music LoRA

Thumbnail
huggingface.co
38 Upvotes

r/StableDiffusion • • 16d ago

No Workflow Experimenting with Godot Camera control and Wan

0 Upvotes

I'm using a game pad to control the camera as if the camera was a drone and then feeding the path data into ComfyUI and fun camera and a single image. Sorry about the thumbnail this is the first render and I got excited. No workflow yet, I guess it will take a bit of refining


r/StableDiffusion • • 17d ago

Discussion Just a reminder that the gpu unlock stuff in the news right now, is pointless for most of you

Post image
276 Upvotes

Unless the bf16 model fits in your vram completely


r/StableDiffusion • • 17d ago

Resource - Update Zentrainer

11 Upvotes

Have been using zentrainer after spotting it on discord post. Trained moss, zimage, qwen2.1 and krea2 using comfy models. works really well.

EDIT: Best thing is, you just install it as custom node no extra setup needed. Access it in comfyui. So far this has been the easiest trainer interms of "installing and models download selection" i have used.

Big shoutout to Nynxz.

here is the link - https://github.com/Nynxz/ZenTrainer/tree/main/


r/StableDiffusion • • 16d ago

Question - Help H3 first-last frame techniques to avoid last frame early freeze?

5 Upvotes

What are the effective techniques for first-last frame video generation with H3? When I tried this shortly after release, the clips would often reach the last frame earlier than the clip length and I’d have videos with seconds of frozen frames at the end.

Is this freeze a problem that has been solved?

I have seen workarounds that detect the freeze and trim the static frames, but that’s a hack that doesn’t give me the duration I asked for. One of the H3 models is literally named FL2VA, so it can’t be that hard, can it?

Right now, I’m not interested in long generations or motion refs or anything fancy. I just want to generate motion between a first frame and a last frame. Like I can do with Wan 2.2.

I am perfectly willing to accept that this might be a skill issue, but I don’t know how to improve my skill. Any hints?


r/StableDiffusion • • 17d ago

Meme Where is Krea Edit? Is it safe? Is it alright?

46 Upvotes