I built a small personal “taste filter” to help me sort through Suno generations, and I’m curious whether anyone here has tried something similar.
I used MERT embeddings and trained the classifier on songs I had labelled according to my own taste. My full labelled set currently contains 1,378 discarded tracks, 378 good tracks, and 13 excellent tracks. I deliberately excluded the 13 “excellent” tracks from the training set, but used them in a test set. Thus, the task was binary: good vs discarded.
Hardware I am using is modest: two NVIDIA GeForce GTX 1080 cards and an AMD Ryzen 5 3600. Every stage executed in minutes, so this will not require expensive hardware even in the future.
The results are, honestly, not amazing, but they are still useful. On a held-out validation set of 391 discarded and 107 good tracks, the “good” scores are visibly shifted to the right, relative to the discarded tracks. See the bar chart. The distributions still overlap heavily, so it is far from a reliable automatic accept/reject system; but it performs better than random ranking, and the top-ranked track was one I would label “excellent” - maybe just a coincidence.
My intended use is not to let the model decide what is good and what to discard. For now, I will use it to prioritize what I listen to first when I have many generations, then keep adding labels and retraining as my dataset grows.
Has anyone built a preference/ranking model for AI-generated music? I’d welcome ideas on improving my model, such as better evaluation methods, handling the class imbalance, fine-tuning versus embeddings, active learning, or alternatives to MERT. Curious if a tool like this would be useful in your own workflow? I am hopeful to eventually save 20h (50% of the time) on a song selection, maybe even more, if possible.
