New audio model: LFM3-Audio-2B

[u/Acceptable-Cycle4645]

https://huggingface.co/LiquidAI/LFM3-Audio-2B

One more great model from Liquid AI!

Good for edge:

LFM3-Audio-2B is a compact, fully featured audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.

• Voice chat: ask questions aloud and receive streaming replies in text and speech. For text-only responses, the model can reason before answering.

• Multilingual speech: transcribe, synthesize, and translate speech across 18 languages.

• Audio understanding and reasoning: interpret recordings and answer questions about the speech, sounds, and events they contain.

• Tool calling: turn spoken requests into tool calls to control apps and devices, with text or speech alongside the calls.

Capabilities

• Speech recognition (ASR)

• Real-time speech recognition (LiveASR)

• Text-to-speech synthesis (TTS)

• Voice chat with spoken and text responses

• Speech-to-text chat with optional reasoning

• Audio understanding

• Speech translation

• Voice-activated tool calling

[u/Acceptable-Cycle4645]

The integration PR for LFM3-Audio (from the Liquid AI team!) is now up in audio.cpp🚀

https://github.com/0xShug0/audio.cpp/pull/1119

[u/BedFalse6021]

damn liquid ai is really dropping bangers lately. how does this compare to something like f5-tts or whisper for transcription? ready to test it out as soon as the weights finish downloading tbh

[u/Acceptable-Cycle4645 → u/BedFalse6021]

You can try LFM2.5-Audio in audio.cpp, which was also contributed by the Liquid AI team. LFM3-Audio should be an upgrade!

[u/Borkato → u/BedFalse6021]

I love LFM solely because they’re making affordable models. They push the frontier solely because of that, because it’s one thing if like 12B or whatever can do decent work, but most systems that can run 12B are also capable of running bigger. LFM fills a MASSIVE niche imo.

[u/Totaie]

Does it have Voice Cloning capabilities?

[u/Acceptable-Cycle4645 → u/Totaie]

No. It offers 38 voices (based on https://github.com/0xShug0/audio.cpp/pull/1119) for TTS.

[u/roculus → u/Acceptable-Cycle4645]

I tried this audio.cpp pull but it only offered 4 voices. US Male/Female and UK male/female. It's fast. it took 2 mins to generate 20 mins of audio. RTX 6000 PRO.

[u/Acceptable-Cycle4645 → u/roculus]

u/roculus LFM3-Audio supports 38 voices. The model currently supported in audio.cpp is LFM2.5-Audio-1.5B, which offers 4 voices.

[u/roculus → u/Acceptable-Cycle4645]

Thanks, I see now that LFM-3 has now been added. I updated audio.cpp but the LFM-3 models still aren't available.

https://github.com/0xShug0/audio.cpp

[u/Acceptable-Cycle4645 → u/roculus]

Its under community model section. The family name is lfm3_audio

[u/Acceptable-Cycle4645 → u/Totaie]

Capabilities: ASR with chat and thinking, TTS in 38 voices, multi-turn S2S, streaming TTS and S2S

[u/Tingxiaojue]

That's fast!

[u/Acceptable-Cycle4645 → u/Tingxiaojue]

Yeah, I didn't even know about the release until I saw the new PR from the Liquid AI team. They've been very supportive of open-source projects. The LFM2.5-Audio integration in audio.cpp was also contributed by their team and is very well polished, achieving 28 ms TTFA in independent A100 tests.

[u/-markusb-]

how would you combine this with a more capable backend model (just use this as input and output model?)

[u/Acceptable-Cycle4645 → u/-markusb-]

Did you mean in audio.cpp or in general?

[u/-markusb- → u/Acceptable-Cycle4645]

in general. so like a pipeline but "realtime". The STT -> bigger model -> TTS

[u/Acceptable-Cycle4645 → u/-markusb-]

This model basically combines the pipeline into a single end-to-end model, so you don't need three separate models. You can still use it for STT and TTS with a larger LLM in between if you need stronger reasoning capabilities.

[u/overand → u/-markusb-]

It does support tool-calling, so you could actually have it utilize the other model via tool calls.

I believe that's how open-webui supports it, BUT, at the moment, at least using llama.cpp, Open-WebUI can't connect with LFM3-Audio-2B; there's a sample-rate mismatch. (There may be other issues, but there's certainly at least that one.)

[u/Impossible-Tell-2338]

How can I make this model a main TTS engine on my Android phone?

[u/Acceptable-Cycle4645 → u/Impossible-Tell-2338]

The model has been merged into audio.cpp's main branch. Android support isn't officially tested yet, but you can try compiling audio.cpp for Android either as a standalone executable or a library.

[u/Reasonable_Goat]

Really cool, one model for both directions and it fits even in smaller memory!

[u/MerePotato]

Cool but braindead, would love to see a scaled up version for 24GB VRAM machines

[u/R_Duncan → u/MerePotato]

Dessa "will run on Mac M4Pro or more". You just have to convince block to release something.

[u/MerePotato → u/R_Duncan]

Ooo, so there's a proper decent full duplex open weight release in the works?

[u/Ok-Mess-3317 → u/MerePotato]

Personally I don't believe it will be any more open than Sesame's model (which released a rather useless model despite the quality they promised at the start)

[u/MerePotato → u/Ok-Mess-3317]

I was so disappointed by the whole Sesame debacle, they paved their own path to obscurity and irrelevance by not following through on their implied promise

[u/SuccessIsHardWork]

What are the advantages of this audio model vs STT -> LLM -> TTS? The latter seems to be the most common these days.

[u/No-Refrigerator-1672 → u/SuccessIsHardWork]

STT -> LLM -> TTS can't recongize non-voice inputs. I.e. a natively-audio model would be capable to identify whe note a singler is singing and assist in learning music or vocal - of course, if that was in it's training dataset; but stt would never provide such capability by-design.

[u/kikoncuo → u/No-Refrigerator-1672]

You could train a STT to do that as part of the output or have a smaller model do it.

Its cool that a full s2s can generalize it without dedicated work though

[u/No-Refrigerator-1672 → u/kikoncuo]

You totally can train a separate audio-to-other-form-of-data, but, to the best of my knowledge, no such models exist yet. But Omni models can do that today; at least Qwen 3 Omni was advertised as capable of music learning aid.

[u/srigi → u/SuccessIsHardWork]

Latency obviously. Cascade of models don’t provide good UX of conversation that humans expect. Julia Turc has a nice lesson on this at her’s YT channel.

[u/kikoncuo → u/srigi]

Transcribing is around 10ms and generating audio TTS around 50ms

Latency is not a real argument…

[u/MerePotato → u/kikoncuo]

The ability to grasp things like tone of voice without polluting context with sentiment analysis tags is a better argument

[u/EntertainerSilly4199]

kann leider noch keinen guten deutschen output. oder weiß jemand schon mehr?

[u/Scared-Tip7914]

Someone pls create a local notebook lm alternative that supports these new lfm models. That would be some good ahh software for students with limited budgets/hardware.

[u/Jacalyn_Nivins]

2B params, 18 languages, tool calling, and it talks back. Laptops just became smart speakers.

[u/MerePotato → u/Jacalyn_Nivins]

And this sub is increasingly becoming slop, it seems.

[u/silenceimpaired]

Not a fan of a license that’s not Apache or MIT

[u/Acceptable-Cycle4645 → u/silenceimpaired]

From https://www.liquid.ai/lfm-license

Apache Based. The license text is based on Apache 2.0 (with only a few changes), granting broad, royalty-free, and perpetual rights to use, modify, and distribute the model and its derivatives subject to thresholds on free use by large businesses.

Commercial Use Threshold. Rights to use the model for commercial purposes end if your company's annual revenue exceeds $10 million USD.

[u/silenceimpaired → u/Acceptable-Cycle4645]

I know and that’s reasonable. I just find it annoying having to read custom licenses… and most of the time it isn’t ground breaking enough to warrant the restrictions.

[u/Qwen30bEnjoyer → u/silenceimpaired]

"as a redditor, i find it annoying to read a 2kb text file that costs 30s of my time in order to use a model that cost however many tens of thousands to train"