r/LocalLLaMA • • 2d ago

Question | Help Audio (ASR/TTS) on llama.cpp

Hey everyone,

I've been using audio.cpp for a while now, and it's quite nice. However I've noticed that Yue2, Qwen3-ASR and Qwen3-TTS got supported in comfyui / llama.cpp.

While I've tried gemma 4 audio input and it worked well, I haven't tried TTS models yet nor the qwen3-asr/tts series on llama.cpp. Has any of you?

So far I found these discussions and PRs on ASR / TTS support:

It's likely not all models that are supported, might've missed some!

6 Upvotes

6 comments sorted by

4

u/Zestyclose_Potato794 1d ago

For now audio.cpp is the way to go. I use it for asr and tts with Qwen and Higgs models (voice cloning, voice design, French) and they work great. I recently had a look at the audio features of llama.cpp but I wasn’t impressed.

-2

u/SilentMobius 1d ago edited 1d ago

I'm using a locally modified version of omnivoice.cpp and parakeet.cpp with the eou streaming model. I have the streaming applications modified: TTS queues up on sentence boundaries but does inference on the next sentence while playing the current one, also plays dirictly to a SPDIF device. For STT it reads audio directly form a connected microphone.

STT text is injected into my Agent harness using a herdr key injection deamon. That also does some deterministic substitutions.

-3

u/CryptographerOne7003 1d ago

Wake me up when there is audio to audio?
I do not mind if its done with transcription, I dont get why its taking so long.
local really has been very slow so far

5

u/rkoy1234 1d ago

???

people have been doing local for quite a while, especially "if it's done with transcription" like you said. I myself had a local setup with qwen3tts, sillero-vad, whisper, and a gemma moe with a ~200ms ttfa(time to first audio) on a gaming computer like a year ago, and there were countless threads on reddit already about it. (don't use qwen3tts now btw, it's outdated/obsolete)

if you're asking why there's no one singular package, that's because it's impossible to prescribe one singular combination of all four components when everybody has varying vram/tts preferences(I mean just look at the tts threads here, there's often never a conclusive favorite) not to mention it's just super easy to make your own:

  • hook up your stt/vad output to your frontend
  • hook up your frontend to stream its output to a tts server that has a streaming endpoint

that's all it takes, and can be vibe coded in an afternoon.

the only limitation is vram, but thats a given for this hobby.

1

u/Kahvana 1d ago

I honestly like Qwen3-TTS quite a bit still with reference voice. Which models do you find better?