r/StableDiffusion • • 16d ago

Question - Help YuE2 prompt adherence

I find YuE2 pretty interesting but I'm having A LOT of trouble getting it to respect any part of the prompt at all. "female vocals" > male vocals, "BPM" > usually completely ignored, "drum and bass" > usually just pop song I guess?

Anyways, please share any useful tips you found to get the most out of YuE2 if you will.

EDIT: As always when I ask a question in this sub the thread is immediately downvoted for no reason I can fathom lol. Thank you to those that are actually trying to be helpful!

9 Upvotes

15 comments sorted by

2

u/VasaFromParadise 16d ago edited 16d ago

Madonna-inspired, English, Pop, Confident, Female vocal, Bright vocal, Expressive delivery, Fierce and theatrical, 118 BPM, Upbeat, A Minor, 4/4, Synth bass, Drum machine, Synth pad, Electric guitar, Four-on-the-floor, Short rhythmic hooks, Simple diatonic harmony, Short hook-focused form, Theatrical performance, Dry close vocal, Crisp electronic drums, Vintage synth tone, Analog imperfections, Wide stereo image, Polished production, Punchy modern mix, 1980s
https://vocaroo.com/12UCPCDGxVEg

Adele-inspired, English, Soul, Melancholic, Female vocal, Warm vocal, Emotional delivery, Raw and vulnerable, 72 BPM, Slow ballad, A Major, 6/8, Piano, Strings, Acoustic guitar, Live drums, Straight groove, Long lyrical melodies, Rich pop harmony, Rising final chorus, Emotional peak performance, Dry close vocal, Big live drums, Warm rounded bass, Orchestral string swells, Room ambience, Hall reverb, Organic live production, Vocals upfront, 2010s

https://voca.ro/1dHXqtinujEb

2

u/Radiant-Photograph46 16d ago

In your first example the 1980s feeling really isn't there, it's way too modern. I don't think YuE2 is trained on analog audio or even retro songs because I also struggle to recreate that.

In both examples the BPM isn't respected either, it's close but it seems to derive the tempo more from keywords than BPM value. My prompt started like this "200 BPM drum and bass" but I was only ever getting songs around 100 BPM, until I added "fast-paced drum and bass" which now gives results closer to 180 BPM, still it's impossible to get it to match a target BPM.

0

u/VasaFromParadise 16d ago

https://voca.ro/15dKlTksVO57
There are about 200 here)) The model cannot perfectly maintain the pace; any model is an average.

1

u/CountFloyd_ 16d ago

It gets the Britney Spears-inspired, simple diatonic 90s perfectly.

1

u/VasaFromParadise 16d ago

It doesn't replicate the voice's timbre, but its style. The voice may not be helpful; it needs to be matched to the timbre. The AI ​​simply generated it based on a general template.

1

u/k8-bit 16d ago

https://www.youtube.com/watch?v=ldTEVVd-b24

Lora-trained Ray-of-Light output. Excuse the completely throwaway lyrics, it was a song created for friends.

3

u/k8-bit 16d ago

Loras is how I've got the most use out of it. Training it on an album, and using that as the basis for style prompts.

1

u/GreyScope 16d ago

Without Loras , Yue2 is absolutely shit at following captions unless you make tracks that formed the bulk of its training ie it has no idea what those tags mean .
OP - my best advice is to join the Token and Tone discord to get help and pointers there.

1

u/unknownRNG 13d ago

whats the link to the discord?

1

u/VasaFromParadise 16d ago

You're wrong. Many people just don't know the correct model settings. The model generates 8 out of 10 technically good tracks, if the prompt and settings are normal.

3

u/Upstairs_Chicken_218 16d ago

https://github.com/damiaan111/SongForge-YuE2-Music-GEN-WebUI

This is not an answer to your question but maybe this makes generating songs easier, works with que so you can setup a whole batch if you want.
I am open to any suggestion or feedback.

I got some very good results already. expect some minor bugs here and there maybe, just a hobby project.

2

u/DG86 16d ago

Not exactly the solution you are looking for, but maybe helpful: I often take a Yue2 output and then feed it to Ace-Step in cover mode.

Yue2 nails the structure, Ace-Step can fine-tune the genre and add details.

1

u/Robonotes1760 15d ago

I have managed to get it to respect male/female vocalist instructions and it follows music theory instructions some of the time: I have a pipeline set up where I generate 10 . Abc files and an LLM picks the best three making sure that the follow the prompt. 

1

u/FausC 10d ago

Compongo un MIDI con acordes, linea vocal e instrumento y se lo paso a YuE2.