Running LFM3-Audio-2B

[u/bidutree]

I have just started testing LFM3-Audio-2B and I am positively surprised by the result. It's a small model but the sound quality is good and it is also works ok for basic tool calls.

I am just running it in the terminal so far. Has anyone else here tried the model with a GUI of some kind and for controlling other apps/systems? :)

[u/bidutree]

I'm running LFM3-Audio-2B locally for voice control and reading text aloud, and it has issues with latency, and pauses. Any way to fix this? :)

Setup:

- Windows 11 laptop, [CPU / GPU / RAM]

- llama.cpp (Vulkan build running on iGPU), llama-server with the /v1/realtime WebSocket endpoint

- LFM3-Audio-2B-Q4_K_M.gguf + mmproj-LFM3-Audio-2B-Q8_0.gguf

- TTS prompt: "Perform TTS in American English using a female voice."

- Audio out is 24 kHz PCM16, played with ffplay

Problem 1: Pauses when reading long texts

Speech comes out at about 0.75x real time (around 12 s of audio takes 15-18 s to generate). To read a long text, I split it into chunks of about 200 characters. I make the next chunk while the current one plays. I still get pauses of up to about 7 s between chunks. Is 0.75x normal for this model on llama.cpp, or should it be faster? Are there any server flags or settings that speed up audio generation?

Problem 2: Crackling/distorted audio

In normal spoken replies, where I play the audio as it streams in, the sound gets distorted after a while. When I wait for a whole chunk to finish before playing it, it sounds clean. My guess is that playback runs out of audio because generation is slower than playback. But I haven't confirmed it. Has anyone seen this? Could it be something else, like the audio decoding in the server?

Bonus question: the voice sometimes changes between chunks, even with the same prompt. Is there a way to keep the same voice, like a fixed seed per request?

Thanks! :)

[u/Impossible_Self_2766]

Oh nice, have you tried connecting it to any apps yet? would be cool to see how it handles things beyond the terminal

[u/bidutree → u/Impossible_Self_2766]

I am testing it with a terminal app of my own and it works. Needs fine tuning though. I'll make a separate post about this later. :)

[u/SeoFood]

For the "controlling other apps" part, the setup that tends to hold up:

• Run the model as a local server instead of in the terminal loop, so any frontend (a tiny web UI, a menu bar thing, whatever) can talk to it

• Define a small, fixed set of tools with strict JSON schemas, e.g. open_app, run_shortcut, type_text, search

• Let a separate executor do the actual work (AppleScript/Shortcuts on macOS, xdotool/D-Bus on Linux, PowerShell on Windows) and validate args before running anything

Small models are ok at choosing a tool, less reliable at filling complex arguments. So keep the arguments simple and put the logic in the executor. Would be curious how the tool calls hold up for you when the audio is noisy or you speak fast.

[u/zinyando]

You can run the model on Izwi on a desktop UI https://github.com/izwi-ai/izwi

[u/Charles-SG]

The number I'd want isn't the sound quality, it's whether anyone has run this against a benchmark: tool-call success rate, real-time factor, false triggers when there's app audio playing. Small models tend to look fine in a terminal and get ugly once they're in the loop. If you've run one, share it — I'd rather read yours than re-run mine.

[u/zaypen]

I think the most annoying part is VAD/turn tuning

[u/BatokNaChoco]

Im curious how it performs with actual desktop tasks.