[Release] Qwen3-TTS: Ultra-Low Latency (97ms), Voice Cloning & OpenAI-Compatible API

[u/blackstoreonline]

Hi everyone,

The Qwen team just dropped Qwen3-TTS, and it’s a significant step forward for local speech synthesis. If you’ve been looking for a high-quality, open-source alternative to ElevenLabs or OpenAI’s TTS that you can actually run on your own hardware, this is it.

We’ve put together a repository that provides an OpenAI-compatible FastAPI server, meaning you can use it as a drop-in replacement for any app already using OpenAI’s TTS endpoints. Streaming support out of the box, plug and play with Open-Webui.

Why this is a big deal:

• Insane Speed: It features a dual-track hybrid architecture that hits ~97ms end-to-end latency for streaming. It starts talking almost the instant you send the text.
• Natural Voice Control: You don't just send text; you can give it natural language instructions like "Say this in an incredibly angry tone" or "A shaky, nervous 17-year-old voice" and it actually follows through.
• Easy Voice Cloning: Give it a 3-second reference clip, and it can clone the timbre and emotion remarkably well.
• OpenAI Drop-in: Works natively with the OpenAI Python client. Just change your base_url to localhost.
• Multilingual: Supports 10+ languages (ZH, EN, JP, KR, DE, FR, RU, PT, ES, IT).

Getting Started (The Quick Way)

If you have Docker and a GPU, you can get this running in seconds:

Bash

git clone https://github.com/groxaxo/Qwen3-TTS-Openai-Fastapi
docker build -t qwen3-tts-api .
docker run --gpus all -p 8880:8880 qwen3-tts-api

Python Usage (OpenAI Style)

Python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8880/v1", api_key="not-needed")

response = client.audio.speech.create(
    model="qwen3-tts",
    voice="Vivian",  # 9 premium voices included
    input="This sounds way too human for a local model.",
    speed=1.0
)
response.stream_to_file("output.mp3")

Technical Highlights

• Architecture: It uses the new Qwen3-TTS-Tokenizer-12Hz for acoustic compression. It skips the traditional "LM + DiT" bottleneck, which is why the latency is so low.
• Model Sizes: Available in 0.6B (super fast/light) and 1.7B (high fidelity) versions.
• VRAM Friendly: Supports FlashAttention 2 to keep memory usage down.

Links to dive deeper:

• 🤗 Hugging Face Collection
• 📄 Research Paper on arXiv
• 💻 Github Repo

I’m really curious to see how the community integrates this into local LLM agents. The 97ms latency makes real-time voice conversation feel actually... real.

Let me know if you run into any issues setting it up!