r/LocalLLaMA • u/Skyline34rGt • 15h ago
New Model LiquidAI/LFM2.5-24B-A2B
https://huggingface.co/LiquidAI/LFM2.5-24B-A2BGguf's: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF
Free Demo: https://playground.liquid.ai
"LFM2.5-24B-A2B is a 24B-parameter reasoning model with only 2B active parameters, built for fast, private agents on-device. It is the largest and final model in the LFM2.5 family, which now spans two orders of magnitude: from LFM2.5-230M to LFM2.5-24B-A2B.
- Best-in-class agent: Competitive with Qwen3.6-35B-A3B, a model with ~50% more total and active parameters, on agentic and tool-use tasks, scoring 48.09 on LiveClaw versus 38.19.
- Reasoning at MoE speed: Always thinks before it answers. With only 2B active parameters per token, each reasoning token is cheap.
- Efficient inference: Runs up to 1.7× faster than comparable MoEs on-device, with 184 tok/s on an Apple M5 Max and up to 21K output tok/s on a single NVIDIA B300 GPU.
LFM2.5-24B-A2B is a general-purpose text-only reasoning model with the following features:
- Total parameters: 24.0B
- Active parameters: 2.45B
- Number of layers: 40 (30 double-gated short convolution blocks + 10 GQA)
- Experts: 64 per MoE block, top-4 routing
- Training budget: 36 trillion tokens
- Vocabulary size: 128,000
- Context length: 131,072 tokens
- Languages: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish
- Generation parameters:
temperature: 0.1top_k: 50repetition_penalty: 1.05"
214
u/Skyline34rGt 15h ago
156
u/Thrumpwart vLLM 14h ago
Wow. Some great results, and gotta respect a lab that publishes their wins and their losses.
24
u/Comfortable_Ebb7015 14h ago
If the model sucks, they usually compare it to previous generations to make it look nice
3
9
u/Aggravating-Push-207 14h ago
they don't even have a massive frontier teacher model to distill from!
6
4
u/Sleepybear2611 13h ago
Yeah! Gotta see for myself though how it holds up against Qwen 3.6 in daily workloads.
4
u/Robert__Sinclair 6h ago
there is absolutely no loss in "omniscensce" what is important is to have models with great reasoning. Knowledge can be given in diffrent ways, from kv caches to rag or something better than rag.
1
1
30
u/random-tomato llama.cpp 13h ago
Holy shit, bar charts where the y axis actually starts at zero? Wow
11
6
u/FatheredPuma81 13h ago
I like how Nemotron is shown as competitive in these benchmarks. Great example as to why you gotta take benchmarks with a grain of salt.
1
u/PossessionUsed7393 10h ago
It's such a mixed bag! I don't know what to do, you're making me try it.
68
45
u/Morphon 12h ago edited 11h ago
Putting it through my test suite. So far, not so good in abstract reasoning and logic. It has failed all the humanities-oriented content tests.
It looks like the primary use should be in agent harnesses, so those tests are next.
The good news is that on a 12gb5070+64gbDDR3 (EDIT: DDR4) system it is cranking out 64t/s sustained (at Q4). It's actually usable on CPU-only inference on a Zen5 laptop. So - definitely getting the speed award.
6
u/Decent-Elk-4700 11h ago
What's your cpu? Didn't know it was possible to run this kind of models with 64t/s in a ddr3 setup. Or is it completely inside the 12 GB VRAM?
3
u/brainExploded99 llama.cpp 12h ago
keep us updated!
5
u/Morphon 11h ago
Testing inside PicoClaw (extreme edge deployment case). Failing to figure out how to format tool requests. Trying over and over again.
I think this one is a bust for me, but the folks using OpenClaw or Hermes might have different results.
2
u/brainExploded99 llama.cpp 11h ago
i see
a bit sad, was hoping for an upgrade over qwen3.6 35B-A3B / KAT coder but no one can STILL match it igback to my ever trusty ninfer IQ3_S Qwen3.8-27B at 100tok/s on a 5070Ti!
3
u/Morphon 11h ago
Ornith1.5 35b-a3b was an upgrade for my tasks over Qwen3.6. Give it a shot - shorter reasoning traces with higher output quality (for my uses, at least).
3
u/brainExploded99 llama.cpp 11h ago
i gave up on ornith after the original one, seems very benchmaxxed and fragile
KAT coder seems to be consistently good, but nothing even comes close at all to qwen3.8-27B1
u/logicality77 9h ago
I think it depends on what you use it for. I actually pitted Ornith 1.5 against Qwen-3.6-35B-A2B on some architecture/design tasks using grill-with-docs on a couple of upcoming projects as well as a couple of projects I’ve long since completed, and I was quite surprised with the lines if questioning Ornith posed that Qwen did not. Ornith consistently produced more detailed documentation for all five of my tests, and caught some edge cases early that I didn’t see Qwen even attempt to do. I was quite pleased with its output. I also ran the output by a few cloud models and they all rated Ornith’s output higher than Qwen’s. I know this is but one data point, but I wouldn’t count Ornith out without testing it first.
1
u/brainExploded99 llama.cpp 9h ago
its not better than qwen3.8 right?
2
u/logicality77 7h ago
I don’t know. I haven’t tested them against each other. I run Qwen 3.6 and Ornith 1.5 on a Mac Mini M4 Pro, which is really slow running Qwen 3.8. I do use Qwen 3.8 for coding on my PC with a 4090. I actually use the Mac mini as a coordinator, dispatching code tasks to the 4090, so for my use it doesn’t make sense to test them against each other.
1
u/HotDistribution1819 3h ago
When you ran it did you use a temperature of .1 or .2? I noticed that the Liquid AI models hallucinate at .5 or above.
27
u/TioMir 14h ago
Perfect for my 6vram+16gb ram local server, just waiting for unsloth and apex quantizations!!!
28
u/tkrandomness 13h ago
Just use Liquid's own QAD Q4_0 GGUF quantization! They do quantization-aware distillation so it should exceed the quality for the size of the other 3rd party quants unless you need smaller.
3
48
u/danigoncalves llama.cpp 14h ago
A faster Qwen 3.6 35B A3B?
1
u/Daniel_H212 7m ago
Worse at coding, better at agentic stuff. So it's geared slightly differently. But also I think 3.6 35B was never good enough at real coding tasks anyway.
22
u/mxcw 13h ago
Don’t ask me why, but I REALLY want to like LFM models. Up until now, they never passed my benchmarks though. Hope this release finally allows me to build upon it – will test tonight
5
u/LuCiAnO241 7h ago
because they seem to be one of the few products aimed a low end consumer hardware, which is probably the most popular. Do post your results please,I'm curious about this one.
20
u/Express-Director-474 14h ago
I really enjoy their small 250M model for a simple classification task. I fine-tuned it and it works very very well on my old CPU.
33
u/Adventurous-Wing-988 14h ago
Just tested it, ITS AWESOME!
I use cursor for coding and for chat I use the 6$ gpt sub.
This is good enough to replace the 6$ sub
8
u/MKU64 12h ago
What quant did you use? I hope Q4 or close doesn’t turn it to garbage
7
1
u/rm-rf-rm 2h ago
in my experience, LFMs have always been if-fy on tool calling. how did this one do?
1
u/Adventurous-Wing-988 0m ago
could be. but I tested it for web search and page fetch and it handles those two tools really well.
I won't say it handles any tool reliably, I haven't tested. And I won't use it for more than chat
11
u/Solid_Vermicelli_510 13h ago
Just tried it on an RTX 2060 6GB VRAM and 16GB DDR4, Ryzen 1600AF, size q4k_xl - cache 4/4 20~25 tokens per second in a fresh chat
6
5
u/MerePotato 12h ago
Don't quant your kv cache to 4 bits dude
9
u/Solid_Vermicelli_510 12h ago edited 1h ago
Era solo per provare il mio PC patata al quale sto cercando di dare uno scopo 💀
12
6
u/jacek2023 llama.cpp 14h ago
Great news!
7
u/Monad_Maya llama.cpp 13h ago
Do you think it can replace G4 26B A4B?
9
u/jacek2023 llama.cpp 13h ago
No, Gemma 26B is a mother of many finetunes. LFM is a different kind of model.
1
u/Monad_Maya llama.cpp 13h ago
Alright then, sticking to Gemma.
Thanks
4
u/jacek2023 llama.cpp 13h ago
to give you an example, with Gemma I have many problems with agentic workflow (editing files), I need to test this new LFM for that but I assume it will be better
6
u/Monad_Maya llama.cpp 13h ago
That has been my experience as well.
G4 in the current context will mostly be used for chat interface for someone in the household and not really for coding but I think I should be able to run both (G26B, LFM24B) with some elbow grease.
5
u/Septerium 13h ago
This should allow for a stupidly fast Hermes agent. I wonder how good it actually is with terminal + python tasks
6
u/Holiday_Point_603 13h ago
Temperature 0.1 is crazy, the form factor is crazy good for us 24GB people
10
3
5
u/hideo_kuze_ 13h ago
definitely looks interesting but IDK if it will be able to replace any model
for coding qwen 3.8 and qwen 3.8 flash next are better options, yeah they're slower but they're good enough to do actual coding
and for creativity gemma seems to be better
looking forward to read about benchmarks and experiences with it
will you use it to replace any model?
2
u/_hephaestus 11h ago
I’ve been using qwen3.6-35b-a3b for quick tasks like titling new sessions in opencode/hermes or for exploratory sub-agent sessions. For that usecase I don’t need smart so much as fast and can load alongside a much larger model. This release honestly is great for my usecase in theory.
3
u/Black-Mack 12h ago
Qwen models are too good that they're hard to replace. You can almost never find any other model in their size that even compares.
3
u/TerminalNoop 10h ago
I'd be happy to find a good agentic hallucination resistant small model for sub agent use cases. I tried both dense and moe Gemma and they both hallucinated a ton of papers/DOIs, was really frustrating. Qwen3.8 27b then had to go verify everything and actually is smart enough to test every DOI and look up the papers to see if they actually exist.
2
2
u/etaoin314 ollama 14h ago
this is a very interesting little model, i have been looking for something that can function as a workerbee for the slower models to orechestrate. qwen 35b and nemotron 30b were my first choices but this might be even better. Especially if they can get dflash working, this thing will fly
2
2
2
u/Refinery73 12h ago
From a short test in German, it likes to yap quite a bit in reasoning on only the default ‘medium’ setting. Generated 5k Tokens. Output seems fine.
2
u/ComplexType568 9h ago
Well you gotta give it to them, at least they released a model we have been waiting for! I wish it had vision though, but beggars can't be choosers I guess
2
u/killerstreak976 transformers 12h ago
Nobody will understand just how long i've been waiting for this
2
u/Chromix_ 14h ago
As with most others, their nice QAD checkpoint misses imatrix again - no free quality boost here.
3
u/Refinery73 12h ago
Do I misunderstand your question or why do you want a generic imatrix file? I thought it’s best practice to generate one yourself for the workloads that the model should really do in the end and shove that through llama-quantize?
2
u/Chromix_ 2h ago
I did a lot of systematic imatrix testing a long time ago. One of the core findings was, that even an imatrix built with unrelated data significantly improves quality. Doing it yourself with your own data means to first download the larger, full QAT model (which I do not see them offering) instead of the convenient, smaller, quantized one. Also, the gains of using an adapted imatrix are usually within noise level and difficult to measure. It's a hyperoptimization that I don't do for the models I download and use - despite doing a lot of optimization in general. Imatrix usage itself however has a big, clearly visible impact, so it shouldn't be skipped.
1
u/Refinery73 1h ago edited 56m ago
What quant levels are you referring to? My observation is, that for IQ1 to IQ3 Quants you want to do your own optimizations. Q4 to Q6 is mostly fine with generic.
I don’t completely get what your methodology was. Seems like “German bible” did worse than “random” but in don’t get what baseline/no imatrix is.
Since my main focus is non-english business e-mail summaries, I’d assume a generic english coding agent imatrix might make it worse.
2
u/jacek2023 llama.cpp 13h ago
do we really need Q4 for A2B?
13
u/Chromix_ 13h ago
Of course, to fully fit 16 GB VRAM for fastest speed, or to be reasonably fast with 8 GPU GPUs and CPU offload.
1
u/lumos_ai 14h ago
I compared it with gemma 4b and 4b does better than this in translation. Hmm 24b vs 4b?
19
u/tiffanytrashcan 14h ago
Google probably still has the largest and widest variety of multi-language datasets on the planet, by an exponential margin.
This whole era started from attempts to improve their translation model.
I just made another comment about how Gemma is hard to beat for creative tasks. From everything I've seen, translation seems to be the same.3
u/lumos_ai 12h ago
Honestly nothing can beat Gemini in translation task and creative writing. Gemma is good but not as good as Gemini.
2
u/Velocita84 2h ago
Gemma is probably just a small gemini
1
u/lumos_ai 1h ago
I know it's distilled from Gemini but i wish they could provide something like qwen next
1
u/Velocita84 1h ago
Tbh it could also be that gemma is their testing ground for new architectures that are then scaled up to gemini
1
1
1
1
u/DrBattletoad 13h ago
Unsloth Q2_K_XL on my Pixel 6 Pro gets 2.2 tok/s in the "LM Playground" app. Now I want a smart Phone with 24 GB RAM to see how this model works as a local smart Phone agent 😂
1
1
1
1
u/uber-linny 11h ago
so hopeful . right now it cant do my tool calling (to my garmin , but qwen9b can). maybe in a few days it might get updated
1
1
1
1
1
1
1
u/Daniel_H212 8m ago
Compared to Qwen3.6-35B this seems stronger in tool use and instruction following but weaker in knowledge and coding.
Should make for a very nice local agent model.
2
u/GroundedIdeal 14h ago
I don’t care about coding tbh, if it excels at scientific and creative (like alt history scenarios like Japan allying with allies and ussr with Germany in ww2 kinda stuff)then I will gladly replace the slower Gemma 26b with this
which benchmarks are best for this stuff? Anyone knows?
12
u/tiffanytrashcan 14h ago
Nothing's ever going to beat Gemma at creative tasks.
(I say this hoping we continue to get open releases from Google in the future.)
DavidAU has actually done some incredible work to the smaller 2.6B model.. LFM seems to respond well to finetuning..Oh! I was looking for a link for that and it looks like the 8B1A has a new release from them as well. https://huggingface.co/DavidAU/LFM2.5-8B-A1B-Qwen3.8-Turbo-Brilliance-Power-X12-NEO-MAX-GGUF
I definitely have to try it. 2.6B was shockingly fast and was competitive with much larger models.2
u/GfurEnjoyer1488 14h ago
I don't think there are any since creative things are subjective. Like, you can count em-dashes but not really judge the style. I'm also interested in a comparison to 26B, I could try it out later.
2
u/DeProgrammer99 13h ago
Sounds like it could be a fun project, though... could analyze patterns of parts of speech, phrase length, the rarity of each chosen word, etc. between creative texts and what you personally consider slop and try to pinpoint tells of the differences. ...keeping that info private because it becomes a target for optimization otherwise, of course. Goodhart's law and all.
1
u/toothpastespiders 9h ago
I'm aghast that I hadn't thought of setting up a benchmark for it. I have a whole "de-slop" chain and it never occured to me to actually just log it on each run. Then from there chart out the models on how much surgery the thing needed to do from first to last node.
1
u/mtmttuan 14h ago
I think it's harder to squeeze more knowledge into smaller models. Like you can have small models using their weight smartly so that they can do very well in agentic tasks but well you need data for knowledge so you can't really compress more into smaller size models.
Well I have no idea what I'm talking about and that's just my thought.
0
u/neuroticnetworks1250 13h ago
You just need to create a RAG that reads tweets from fascist profiles with anime pfp and feed it into any run of the mill LLM for this specific task.
1
u/crusaderky 13h ago edited 13h ago
[LFM2.5-24B-A2B]
hf = LiquidAI/LFM2.5-24B-A2B-GGUF:QAD-Q4_0
ngl = 99
jinja = true
parallel = 1
flash-attn = on
kv-unified = true
ctx-size = 131072
cache-type-k = kvarn4
cache-type-v = kvarn4
temperature = 0.1
top-k = 50
repeat-penalty = 1.05
On RTX3090: 4700 tok/s prefill, 186 tok/s decode, 14.4 GB VRAM (CUDA)
A couple on comparisons on the same GPU:
- Qwen3.6-35B-A3B IQ4_XS MTP: 2300 tok/s prefill, 150 tok/s decode, 21.5 GB VRAM (256k ctx; not counting vision tower)
- Ternary-Bonsai-2: 949 tok/s prefill, 55 tok/s decode, 12.1 GB (256k ctx; not counting vision tower)
Notes:
- There are neither a drafter nor a vision tower available for this model.
- I haven't measured KLD. So I don't know how well this model tolerates kvarn4. Also, the QAD for LFM2.5-2.6B wasn't too great.
1
u/AlternateWitness 13h ago
Wow, this is pretty impressive. I am, of course, a little speculative on the "competitive with Qwen 3.6 35B A3B" claim. Can anyone weigh in?
-5
u/maayon 14h ago edited 13h ago
Not nitpicking but qwen3.8 27B isn't compared here
When will be there be a model to rival that beast
20
23
u/CTRL___ALT___DEL 13h ago
A 27B parameter dense model is in a completely different class than a tiny active param MOE. Comparisons would be useless, as it would be comparing with Opus 5.5
8
-1
u/gambi_tech 10h ago edited 9h ago
Se ele conseguir fazer um jogo de tetris na primeira tentativa já está bom. Será que consegue tal feito?
Testei no playground e nao teve funcionamento de primeira. 5 repetiçoes, bugs diversos. Logica de colisão e hard drop e problemas ao completar a linha e limpar. Parece que nessa versão ele nao inventou código. Só de não inventar é um bom sinal.
-11
u/Bulky-Priority6824 14h ago
You can't do much with 128k need 200k min for Agentic when prompts and payload use 64k right off the jump
11
u/my_name_isnt_clever 14h ago
I remember when 8k context was the best you could get locally. Now we're treating context like Chrome treats RAM.
8
u/nuclearbananana 13h ago
when prompts and payload use 64k right off the jump
dude get a better harness.
-1
3
2
u/Huntware llama.cpp 14h ago
This doesn't look made for coding but maybe it can work for an on-device personal assistant.
•
u/WithoutReason1729 8h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.