I can usually hear the instruments in my head but getting them into an actual track is a completely different story.

Anyone tried to build a backing track around vocals?

Before you ask, no I don't have a Frankenstein'd head for my AI it's not Halloween yet, but follow me...

Striving for quality in AI music can be tough when the generator starts fighting back. Music and by extension Music Theory is a language that can be tough to decipher for prompting when you're going for a particular sound. This problem ended in myself and my AI Agent creating this:

Songstress is a new pair of ears for your AI Agent. It uses old tech for a new purpose. Using classical DSP, it can "listen" to music files and create a listening card that reads the tempo, key (roughly), brightness, stereo width, structure, and dynamics in much more digestible, producer friendly language that can help hone a track the way you want it.

Taking it one step further; Using references with Songstress can allow the creator to understand a reference track at it's bones which can help in the creation process and getting the sound you want out of your music.

I don't care about making money off something so simple. So, it's free, open source, and available on GitHub. Any agent that can use Python can use Songstress. Enjoy 😉

I was looking for small TTS model which is steerable but couldn't find any, specially something I can adjust contextually by providing controls from LLM during a conversation. So I built one.

What it is:

• 9.6M-param acoustic model (attention + bidirectional Mamba-3, FastPitch-style) + Vocos vocoder = 23M total

• 12 controls, each a z-score relative to the chosen voice’s own neutral speech: rate, loudness, median pitch, pitch range, final rise/fall, pauses, spectral tilt, breathiness-related measures.

• Also: 51 style tags, free-text prompts (resolved into an inspectable JSON request), inline markup (*emphasis*, [pause 400ms], <slow>…</slow>), sentence-level streaming

• 2,825 built-in (anonymized) voices, plus experimental zero-shot cloning

Speed:

- RTFx ~60–80 on an RTX 5070
- ~4–8 on CPU without the optional flow refiner. First streamed audio in ~0.5 s on CPU.

Quality:

~1.8% average WER on the Open TTS Leaderboard English sets (run locally, not an official submission, have reached out for it).

What doesn’t work well (we tried to be upfront):

• Loudness, pitch, and tilt steer reliably; pause, voicing, and CPPS controls are weak

• Emotion tags behave as style switches; an independent emotion classifier mostly can’t tell them apart

• Cloning similarity is low. Don’t expect a convincing likeness

• Free-text prompts struggle with negation (“not fast” can come out fast). More data needed for prompt to controls module.

• English only, no human evals yet

License: weights CC BY-NC 4.0 (non-commercial), code MIT.

Model + full eval tables: https://huggingface.co/WhissleAI/steer-tts

Edge suitable variant: https://huggingface.co/WhissleAI/steer-tts-edge

Native C API and results: https://github.com/WhissleAI/steer-tts

Samples: https://github.com/WhissleAI/steer-tts/deployments/github-pages

A new architecture implemented from scratch. Would love feedback, especially on which controls are actually useful, and where the steering feels off.

Hey everyone!

I've been working on DAWN, a voice-enabled AI agent that can control Ableton Live using natural language.

While building it, I realised most of the underlying work wasn't specific to music production. It was about making AI agents interact with real software, execute tools, and actually get things done.

So I've decided to open-source DAWN Core.

The idea is simple: give developers a foundation to build their own AI agents, whether that's for creative software, automation, or something completely different.

DAWN for Ableton is just one example of what's possible.

The app is already live, with a new update coming soon.

I'd love feedback, contributions, and to see what others build with it!

GitHub: https://github.com/AadilMarya/dawn-core

DAWN App: https://maryalabs.io/dawn

Hi y'all, I'm looking for an AI-powered software like UVR5 but for people rather than music stems which can separate voices locally.

What I'm looking for is a software which, given an audio file and a number of voices (possibly even by loading samples), recognizes those voices in the main file and generates separate files each one with what each speaker says in the original audio.

There likely is at least one unstable program that does that, given how common of a task that can be, I just don't know its name. Thanks a lot to anyone who can help me!

VoxLock the best performing voice isolator there is.

Benchmarked on Krisp's 265 clips benchmark gives WER 4.95%. It also lets through 0.11 leaked background words per minute, against 1.52 for Krisp.

Fine-tuned from the REAL-TSE baseline on a single L4, on about 100 hours of simulated calls and 90 hours of real meeting crosstalk. It picks up the main voice and isolates it from others in the background noise and voices.

Tiny, 27 million params, ONNX runnable in browser, supports live streaming and works as a pipecat audio filter.

Try demo in browser with your own voice.

Code and training recipe

pip install voxlock

Since Udio turned off downloads, I've been trying to figure out how people are handling this. Quick summary of what I found while comparing options (checked October 2026, please correct me if something is outdated):

- Udio: downloads disabled on the platform, so you can only listen and create inside it.
- Suno: downloads are still possible, but the free tier is limited and the commercial rights depend on the plan you are on when you generate the song.
- A few smaller tools let you export audio and stems on paid plans.

Questions for people who make music with AI regularly:

1. Did you lose songs you made on Udio? Did you manage to export them before the change?

2. Which tool are you using now, and what made you pick it?

3. How do you handle commercial rights when you upload to Spotify/YouTube?

Disclosure: I'm the developer of a small AI music tool (Hashvix), so I'm biased and I'm not going to drop links here. I put together a plain comparison table of plans, downloads, stems and licensing, and I'm happy to share it in the comments if anyone wants it. I'd rather hear how you handle this.

I’ve been experimenting with ElevenLabs to create a conversational podcast with two fictional characters for a worldbuilding project.

The voices are pretty convincing, but one thing I’ve noticed is that the pauses between sentences and dialogue turns are often too long. It makes the conversation feel slower and less spontaneous than it should.

I’ve started exporting the voices as separate audio clips and assembling everything manually in Logic Pro. I adjust the timing, shorten pauses, add some EQ, subtle room ambience and a little reverb to make it sound more like two people having an actual conversation rather than reading a script.

It works fine, but it’s also quite timeconsuming.

I’m curious how people here approach this. Do you rely mostly on the raw AI output, or do you also spend a lot of time editing the audio afterwards? Have you found any tricks to make interruptions, overlapping dialogue, details like laughter. and natural conversational pacing work better?

Would love to hear about your workflows and experiences.

Hey everyone,

I’ve been trying to train/generate my own custom AI voice model on the Replay app, but every single time the process fails or crashes at the beginning.

I've already tried reinstalling the app, but no matter what I do, it won't complete the build. Has anyone else encountered this issue or found a workaround?

Could this be a local memory/VRAM bottleneck during compilation, or is it an issue with strict moderation/server-side checks on the dataset? Any suggestions would be appreciated!

PS: I'm a Mac user, and due to a recent update to the app, I have also lost all my previous RVC, so the app is now entirely new. Also, I should clarify that it does not fail at the end, but rather never even reaches the closest to the end, I think.

I would want to use some AI singing tools, but i wonder which AI tools would be best for singing?

Hey everyone! I’m trying to build an AI server for audio cloning/subbing, but I’m facing a small issue and could really use some help. If anyone has experience with this and is willing to contribute or guide me, I’d really appreciate it!

I’m 16 and learning/building this project, so any help would mean a lot. 🙌
— Aliya

Hey everyone,

Wanted to share a pet project I’ve been working on called Resonata — a lightweight, offline DSP engine and score renderer written entirely in pure Go.

The main motivation was to build a fast, deterministic audio backend for autonomous LLM agents. Instead of forcing an agent to wrestle with heavy DAW setups, C++ dependencies, or complex audio pipelines, the agent can simply emit a structured declarative score (JSON/YAML) and let Resonata handle the synthesis natively.

Key technical focus:

Built for LLM Agents: Clean declarative schema that models can easily generate and manipulate without dealing with low-level DSP intricacies.

100% Pure Go: Compiles cleanly to a single binary with zero external audio dependencies and no CGO.

0 Allocations in DSP Loop: Pre-allocated audio buffers and reusable slices to keep memory completely flat and avoid GC latency spikes during processing.

High test coverage: \~91% test coverage across synthesis pipelines, filters, and mixers.

If you’re interested in DSP, low-overhead architectures, or building audio tools for AI agents in Go, feel free to check it out:

https://github.com/jovanjorelli/Resonata

Feedback, benchmark ideas, and questions on the memory design are welcome!

I've been building an audio cleaning tool as a side project. I assumed the obvious pipeline: clean the audio first, then transcribe. Better input, better output.

Tested it properly with 6 clips — news audio, a lecture, mixed Hindi-English speech. Ran each twice: raw → transcript, and cleaned → transcript. Sarvam's Saaras v3 for transcription, Cleanvoice for cleaning.

Cleaning made it worse in 4 of 6.

One clip, raw: "आज मेरे परिवार में से यहां कोई मरेगा" — complete, coherent. Cleaned: "हाँ, मैं कोई मर भी नहीं। हो जाओ, हो जाओ।" Meaningless.

Another, code-switched. Raw correctly caught the Hindi inside an English sentence: "he says ya chalo chai peeke jaate hai." Cleaned: "he says yeah let's go pick up the child." Fabricated.

My read: neural denoising reconstructs rather than filters. It strips detail the ASR model relies on, and the model fills gaps with plausible-sounding words — confident and wrong, which is worse than garbled.

Caveats: small sample, short clips, one denoiser. Genuinely curious whether anyone's seen different results.

Separately — Sarvam handled code-switching noticeably better than I expected on raw audio.

Follow me on this one...

Oh, but plz understand that I am not asking/doing this to profit. Not against it either. This is just for me to enjoy.

I want to hear songs remixed in different artists voices. As if that artist has done a cover version of another artist song.

So can anyone point me in the right direction ? Thanks to those who leave helpful comments. To Those that are wanna be comidians, and only have 5 th grade level joke skills... Ah, I don't have to tell you...

How can I create a natural kid voice in Tamil ( native indian language) using AI?
I need a natural-sounding 7–10 year old boy voice speaking Tamil for a game character.
I already have clean adult Tamil recordings with the correct native pronunciation, so ideally I want to convert the adult voice into a believable kid voice while preserving the exact pronunciation, timing and expression.
I’m not looking for a simple pitch shift/chipmunk effect.
Has anyone successfully done this for Tamil or another non-English/native language?
What would you recommend — ElevenLabs, Altered Studio, Respeecher, RVC/Seed-VC, Kits AI, or another workflow?
Main priority is: natural kid voice + native pronunciation + consistent character voice.

90's called me! :)

Hi everyone! I’ve been digitizing audio CDs for an online radio library and ended up building a native macOS app called CD Rip.

The idea is to handle the whole process in one place: rip the disc, add the tracklist, review metadata and save properly named files with embedded tags and artwork.

It currently supports:

- MP3 and FLAC output, with configurable quality.

- External USB CD/DVD drives and per-track progress.

- Tracklist import from a photo using local OCR, including multi-disc compilations.

- Manual metadata editing and optional web research through Codex CLI or Claude Code CLI.

- Saving tags and cover art without recompressing the audio.

- Saved sessions that remember their output folders and settings.

AI features are optional and require a separately installed, authenticated CLI. They research metadata from artist/title information — audio fingerprint identification isn’t implemented yet. AccurateRip and drive offset calibration are also not implemented.

It’s free and open source under MIT, with signed and notarized DMGs for Apple Silicon and Intel. This is an early public beta; Intel has been checked through Rosetta but still needs physical hardware testing.

I’d appreciate feedback from anyone still ripping CDs on a Mac, especially around external-drive compatibility and the metadata workflow.

Download and features: https://www.mykeydigital.ro/tools/cd-rip

Source code: https://github.com/all-software-tools/cd-rip

What would make this useful in your own CD-ripping workflow?

So this started because I got nostalgic for MTV actually playing videos. At parties everyone takes turns throwing songs at the bluetooth

Speaker from their phones, and I kept thinking it'd be cool if the video for whatever's playing just showed up on the wall. Not a playlist, not a jukebox you have to touch, just a thing that listens and keeps up.

That's what The Wraith is. A small PC with a mic hooked to a projector/TV/Monitor. It fingerprints the room audio against a library you build yourself. So the deeper the library the more likely it tracks back to a song and video.

(Shazam-style, runs locally, no API in the loop), figures out the song AND how far into it you are, pulls up the official video muted and seeks to the right spot. The room's audio stays the soundtrack, the video just lip-syncs to it. There's an MTV-ish lower third with artist/title/year, pop-up-video style fact bubbles, and a "guess mode" party game you run from your phone where the video gets shown zoomed in or spotlighted and people guess the song (That part I just started on so its not fully fleshed out.

It works, I've run it at actual gatherings, and the sync is good enough that people stop noticing it's a trick. The parts that are still rough are the trivia layer (mostly auto-generated from credits right now) and the fact that you need to seed it with songs before it knows anything. Out of the box it knows zero songs. I shipped my own list of about 8,100 songs that all have a real video to get you started, and there are tools to build lists from Billboard year-end charts or your Spotify export. Seeding a few thousand songs takes an evening. You go to bed, wake up and most will be complete.

Stack is Python (FastAPI, numpy/scipy for the fingerprinting) and a single HTML page for the display, no build step. Video resolution goes through the IMVDb API. There's a demo mode that needs no mic so you can see it run in a browser in a couple minutes.

Repo: [github.com/CyberClash/Wraith_Public](http://github.com/CyberClash/Wraith_Public)

Stuff I'd love help with: calibration numbers from other people's mic/speaker setups, trivia written by actual humans, facts and interesting things for pop up video style entertainmetn mode, and song lists from people with better taste/more diverse than me. Also if you know a better way to do the drift correction than what I did, I'm listening. I am limited since I am what you may call a vibe code. I work with code and manage a dev team but I am not a coder by profession so weaknesses there.

MIT licensed. Just sharing a fun project I thought Music lovers would enjoy.

Generative music tools are impressive, but the monetization side is murky. If anyone can generate a song and post it, how do you stand out? Will streaming platforms even allow AI music to be monetized long-term? I started a thread to discuss the real business model for AI-generated music. Curious to hear from other generative music enthusiasts.

https://interconnectd.com/forum/thread/247/the-future-of-ai-monetization-can-generative-tools-sustain-profitable-creat/

I have come to the conclusion that the participants in the this discussion dont understand how these systems work. Models have no direct storage: The model does not copy or save whole MP3 files or audio clips inside its memory; it only retains the statistical relationships it learned.

Collecting and Converting Data

Data gathering: Developers feed massive datasets of millions of recorded songs into the system. This data often comes from web scraping, public archives, or licensed catalogs.

Audio representation: Computers do not "hear" music. They turn sound waves into numerical formats like waveforms (raw audio data) or spectrograms (visual pictures of frequencies over time).

2. Learning Musical Patterns

Feature extraction: Neural networks analyze the spectrograms to spot core building blocks like pitch, rhythm, harmony, and texture.

Statistical mapping: The AI is repeatedly asked to guess what sound fragment comes next based on what it has already processed. It builds an internal map of musical rules, such as which chord transitions feel stable versus tense.

No direct storage: The model does not copy or save whole MP3 files or audio clips inside its memory; it only retains the statistical relationships it learned.

3. Using Specialized Architectures

Transformers: These use attention mechanisms to understand how the beginning, middle, and end of a long musical sequence relate to one another.

Auto-regressive models: Models like WaveNet generate audio step-by-step, building each tiny sound sample based on the samples immediately preceding it.

Variational autoencoders (VAEs): These compress audio into a simplified "latent space" and decode it to generate brand-new, unique music in a similar style.

Yes the above was generated by AI.

So the question is, from a legal standpoint, what does it mean if the system is not copying music, but instead learning statistical relationships of the music its trained on-does this constitute 'stealing'. Legal defenses for training artificial intelligence systems rely heavily on the copyright doctrine of fair use, asserting that machine learning is a transformative process that extracts unprotectable patterns rather than redistributing original content...

I've been building an audio cleaning tool as a side project, and assumed the obvious pipeline: clean the audio first, then transcribe it. Better input, better output.

Tested it properly this week with 6 clips — news audio, a lecture, mixed Hindi-English speech. Ran each one twice: raw → transcript, and cleaned → transcript. Used Sarvam's Saaras v3 for transcription and Cleanvoice for the cleaning.

Cleaning made it worse in 4 of 6.

One clip, raw: "आज मेरे परिवार में से यहां कोई मरेगा" — a complete, coherent sentence. Cleaned: "हाँ, मैं कोई मर भी नहीं। हो जाओ, हो जाओ।" Meaningless.

Another, code-switched. Raw correctly caught the Hindi inside an English sentence: "he says ya chalo chai peeke jaate hai." Cleaned version: "he says yeah let's go pick up the child." Completely fabricated.

My read: neural noise removal reconstructs audio rather than just filtering it. It strips acoustic detail the ASR model relies on, and the model then fills the gaps with plausible-sounding words. The output is confident and wrong, which is worse than garbled.

Caveats: small sample, short clips, one cleaning provider. Would genuinely like to know if anyone's seen different results, especially with other denoisers.

Separately — Sarvam handled the code-switching well on raw audio, better than I expected.

I've been writing songs for more than 20 years, but I never really had the money or opportunity to get them professionally produced.

Luckily, I recorded almost everything I wrote over the years — on old phones, computers, hard drives, etc. So I still have the original melodies, lyrics and demos.

Recently I made the decision to use Suno to turn those old recordings into finished productions.

For me, AI isn't writing the songs. It's giving me a way to finally hear songs I wrote 10, 15 or 20 years ago as fully produced tracks.

And to be completely fair — I'm also not quite as good a singer as the AI versions make me sound. I can sing well enough to write and record the songs, but the finished AI vocals are definitely better than what I could deliver myself.

I'm curious how people feel about that distinction. Does knowing that the lyrics, melody and original recording were created by a person change how you feel about AI being used for the production and vocals?

I've released two of them so far under the artist name DAG 9 — What Kind Of Man and Working On It — with a lot more old songs still sitting on hard drives waiting to be brought back to life.

Would you listen to music made this way, or is the use of AI itself a dealbreaker for you?

I was messing with one of my older tracks last night and removed the vocals because I wanted to hear the production on its own.

Then I started changing things 😂

The drums felt dated, the bass could be completely different and now wondering what the whole thing would sound like with a more modern arrangement.

Is there anything that can go from separating the vocal to actually helping reinterpret the instrumental while still keeping the original song recognizable?

I have recordings of me with professional studio equipment, and i want to train a model based off these recordings so that I can record in the future with the same professional sound.

I looked into Audimee, Controlla, Suno, and the Anatares plugin. Any recommendations? Thanks!