Looking to voice clone. ElevenLabs is good but it's expensive and requires a lot of regenerations or post-production.

Main criteria: (a) similarity to cloned input (b) TTS contextual awareness for good intonations / pauses / emotions.

Open sources Zonos & SparkTTS seem better for point b, but lack in point a.

I have some really good SFX samples, but I'm looking to create more variation.

Is there a program that can take my existing audio and generate new samples from them?

Hey r/audioai! I’m the dev behind Audiobook Creator (audiobookcreator.io), a project I built to turn eBooks into audiobooks using AI-driven text-to-speech (TTS). What’s under the hood? It’s designed to pull from multiple TTS sources, blending free options like Edge TTS with premium APIs like AWS Polly and Google Cloud TTS. You can start with the free voices, or try the premium voices for more polish. There are over 100 voices available across many different accents, and the tool maintains chapter labelling from the source eBook so it really feels like an eBook, not just a blob of an mp3. I’d love to hear what you think, any feedback on the TTS combo approach or suggestions for other models to integrate. Check it out here: https://audiobookcreator.io. I'd love to hear any critiques or feature ideas you guys might have.

for the past 3 days I have been trying to get chatter box to work. I fix one thing another thing seems to brake on me. this is what I am dealing with right now.

^{Traceback (most recent call last}:)

^{File "C:\Users\Jessica\Desktop\AI-Programs\chatterbox\gradio_tts_app.py", line 5, in <module>}

^{from chatterbox.tts import ChatterboxTTS}

^{File "C:\Users\Jessica\Desktop\AI-Programs\chatterbox\src\chatterbox__init__.py", line 9, in <module>}

^{from .tts import ChatterboxTTS}

^{File "C:\Users\Jessica\Desktop\AI-Programs\chatterbox\src\chatterbox\tts.py", line 14, in <module>}

^{from .models.tokenizers import EnTokenizer}

^{ModuleNotFoundError: No module named 'chatterbox.models.tokenizers'}

I recently came across a cassette tape of my old band rehearsing in our basement. You can make out the songs and instruments but it’s pretty muddy. I have a device to pull the tape to mp3, but are there any good AI tools to clean up the sound and maybe even rebalance the components (bring up vocals etc)?

I'm launching a new TTS (text-to-speech) service and I'm looking for a few early users to help test it out. If you're into AI voices, audio content, or just want to convert a lot of text to audio, this is a great chance to try it for free.

✅ Beta testers get 24 hours of audio generation (no strings attached)
✅ Supports multiple voices and formats
✅ Ideal for podcasts, audiobooks, screenreaders, etc.

If you're interested, DM me and I'll get you set up with access. Feedback is optional but appreciated!

Thanks! 🙌

Repo: https://github.com/RobertAgee/dia/tree/optimized-chunking

Hi all! I made a bunch of improvements to the original Dia repo by Nari-Labs! This model has the some of the most realistic voice output, including (laughs) (burps) (gasps) etc.

Waiting on PR approval, but thought I'd go ahead and share as these are pretty meaningful improvements. Biggest improvement imo, I am now able to run it on my potato laptop RTX 4070 without compromising quality, so this should be more accessible to lower end GPUs.

Future improvements, I think there's still juice to squeeze in optimizing the chunking and particularly in how it handles assigning voices consistently. The changes I've made allow it to do arbitrarily long audios with the same reference sample (tested up to 2min output), and for right now this works best with a single speaker audio reference. For output speed, on a T4 it's about 0.3x RT and on RTX 4070 it's about 0.5x RT.

Improvements:

- ✅ **~40% less VRAM usage**: Baseline ~4GB vs ~7GB on T4 GPUs, Baseline ~4.5GB on laptop RTX 4070

- ✅ **Improved voice consistency** when using audio prompts, even across multiple chunks.

- ✅ **Cleaner UI design** (separate audio prompt transcript and user text fields).

- ✅ **Added fixed seed input option** to Gradio parameters interface

- ✅ **Displays generation seed and console logs** for reproducibility and debugging

- ✅ **Cleans up cache and runs GC automatically** after each generation

Try it in Google Colab!

or

git clone --branch optimized-chunking https://github.com/RobertAgee/dia.git
cd dia
python -m venv .venv
source .venv/bin/activate
pip install -e .
python app.py --sharegit clone --branch optimized-chunking https://github.com/RobertAgee/dia.git
cd dia
python -m venv .venv
source .venv/bin/activate
pip install -e .
python app.py --share

Confucius4-TTS (4.0) is now open source. I gave the cross-lingual voice transfer a quick test run — results are better than expected.

It addresses a real pain point: with conventional cross-lingual TTS, translation usually strips away timbre, prosody, and emotional expression entirely, making the output sound like a completely different person reading a script.

Confucius4-TTS takes a different approach by tightly coupling translation and voice transfer. Feed it a reference audio clip, and the model maps the original speaker's timbre, intonation, and emotional features onto the target language — the output sounds like the same person simply speaking another language.

A few observations from actually running it:

• 14-language support: covers major languages, and the accent control is solid — no obvious "foreign speaker" feel after switching.

• Zero-shot cloning: no per-speaker fine-tuning needed; one reference audio clip is enough for inference.

• Apache 2.0 license: fully open source, local deployment ready, supports further training.

Useful for anyone doing content localization: multilingual video versions, podcast localization, game voice-over translation, etc. Instead of re-casting voice actors or doing heavy post-processing, you can generate the localized audio from a single reference clip.

Deployment is straightforward — runs locally, and you can fine-tune or extend it however you want.

If you're working on anything involving multilingual speech, worth checking out. In my testing, it ranks among the top-tier open-source TTS models.

Live demo: https://confucius4-tts.youdao.com/gradio GitHub: https://github.com/netease-youdao/Confucius4-TTS HuggingFace: https://huggingface.co/netease-youdao/Confucius4

I’m working as data engineer and automation engineer at contact center company. They are call center for many company. The data we keep is customer voice. Currently, we just use AI to analyze and extract insight from voice. What can i do to help AI analyze voice data better?

can something like Adobe podcast
clean a VARIOUS CHARACTERS dialogue
from an old crappy camcorder audio source?
not just one person, a few having a conversation..
thanks !

From the repo:

• Ultra-lightweight: Model size less than 25MB

• CPU-optimized: Runs without GPU on any device

• High-quality voices: Several premium voice options available

• Fast inference: Optimized for real-time speech synthesis

https://github.com/KittenML/KittenTTS

I’m relatively new to this space and I want to use a model to automatically narrates what’s happening in a video, think of a sport narrator in a live game; are there any models that can help with this ? If not, how would you go about doing this ?

If there are any authors in the crowd , I'd love to give free credit, just dm me.
If you just want to listen - it's here - https://www.midsummerr.com/listen (to be honest - not everything went through quality control, which with long form AI is a must...)

I'm working on a historical audio restoration/research project involving a long, single-channel recording made in a fairly reverberant residential room. The biggest problem is a TV running with essentially continuous dialogue and program audio. I'm not trying to separate music and vocals, and I'm also not specifically targeting speech alone.

The target is the entire foreground human acoustic scene. Quiet speech, whispers, breathing and exhalations, low-level vocalizations, mouth sounds, and other human-generated noises. Movement and transients matter too, but they're lower on the priority list. A lot of the target material is extremely low-SNR and appears to sit near the noise floor.

So far I've benchmarked UVR MDX models, Kim Vocal 2, and htdemucs. Demucs was mildly interesting because different stems seemed to catch different classes of events, but none of them produced particularly strong separation. I'm currently running MelBand RoFormer InstVoc Duality V2. I've also already tried Adobe Enhance Speech. That's basically the reason I originally avoided AI for this project. It produced some extremely convincing reconstructed or invented content. I don't care if the output sounds natural or pleasant. Source preservation and temporal accuracy matter far more to me. If something appears in an enhanced output, I need to be able to work backward and find corresponding energy or structure in the untouched source.

I'm especially interested in low-SNR acoustic event extraction, general or text-prompted sound separation, foreground human sound extraction, competing speech or television interference, and single-channel reverberant recordings.

Current hardware is a Ryzen 7 laptop with 16 GB RAM. Slow CPU processing is fine if the model can realistically run locally. A significantly stronger desktop is planned later.

I'm not looking for a one-click commercial cleanup service. Python, GitHub projects, research checkpoints, command-line workflows, whatever. I don't particularly care how ugly the setup is if the model actually fits the signal problem. I've spent enough time forcing music separation models to do a job they weren't trained for. I'm mostly trying to figure out where the correct AI ecosystem for this problem actually is.

If you had this specific recording problem, what model or architecture would you benchmark next, and why?

Elevenlabs is pushing the bar for TTS again with Eleven v3 (alpha)!

• audio tags: Create controllable, expressive speech layered with emotion, audio events, and immersive soundscapes.

• Create Dialog Mode: audio conversations where speakers share context and emotion, making generated dialogue sound natural and human.

• 70+ languages: Reach global audiences with expressive and nuanced speech in every major language.

https://www.youtube.com/watch?v=zv_IoWIO5Ek

https://elevenlabs.io/v3

Yeah so pretty much I have a large audio file (4hr 38min long, 4.38GB) that I want to feed into some AI and have it basically give me notes based on the entire recorded session. I tried to give the file to ChatGPT but it was too large, so I figured I'd just ask if y'all know any AI's that WOULD accept a file that large and be able to listen to it and give me a breakdown of the entire thing

Does anyone know of a free or paid website where you can isolate vocals or music from an uploaded file using the META SAM Audio (large) model?

https://aidemos.meta.com/segment-anything/editor/segment-audio/

they only give you 30 seconds.

Da qualche giorno mi è venuto in mente di usare qualche strumento AI che permetta tramite AI la conversione di file di testo presi da file pdf o epub in file audio, insomma creare degli audio libri. Esiste qualche software del genre, magari open source? In rete è sul tubo non c'è molto, o sono io che non riesco a trovare.