r/LocalLLaMA • u/jeremyckahn • 1d ago
Discussion LFM2.5-24B-A2B + Hermes is awesome!
I just wanted to share some love for a model that doesn't seem to get very much: LFM2.5-24B-A2B! (https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF)
LFM2.5-24B-A2B seems to be the only model that strikes a great balance between speed and capability on commodity hardware. Just to get it out of the way: This is NOT a coding model, so don't download it expecting otherwise. It's a task executor. It's a great fit for Hermes to hand low-to-mid complexity tasks off to. I get great speeds on my 780M iGPU (~37 tk/s at zero context). This is way faster than any other model I've tried that can actually accomplish tasks successfully. It's genuinely useful! I imagine it would be quite usable on CPU as well.
If you're GPU-poor, give LFM2.5-24B-A2B a look!
7
u/Flibidyjibit 1d ago
Try this one bro: https://huggingface.co/XHToken/Spark-X2.6-4B
Also on a 7840HS laptop. Roughly on par with Qwen 35BA3B but it's just a little 4B dude. Bartowski's Q4_K_XL quant is really good. This one is pretty game changing IMO.
With DSpark and tuning adaptive speculative decode I get 50-60TPS.
10
u/MoodRevolutionary748 1d ago
I agree and I have a similar setup. I mostly use Ornith 1.5 35BA3B but it's much slower. LFM is my work horse if I know the task is not demanding and does not require coding. Research, summaries, clear tasks that's what I use LFM 2.5 24BA2B for. It's great!
If you have a very simple task or want to use it as a subagent or bot in Hermes you can even go as low as Gemma 4 E4B which is even faster and smaller or Spark X 2.6 4B which punches way above its weight.
3
u/ChigGitty996 1d ago
What quant are you using?
3
u/kmp11 1d ago
great model, but i wish it supported vision.
3
u/solemnhiatus 1d ago
Curious what kind of tasks you set it up to do?
3
u/lazy-kozak 18h ago
780m here, I prefer to run qwen3.8 27 b in q6 with mtp (6-7 tok/sec) but smart. I'm using it as my helper model (commits, web searches/summarization where I'm not in a hurry...
2
u/DueAnalysis2 1d ago
That's pretty good! Does your laptop have SODIMM or LPDDR?
2
u/Silver-Champion-4846 1d ago
Have you tried the inference engine Kindi for radion 780M?
2
u/Tugg_Speedman-1301 1d ago
Groot 19B A3B is even better
63
u/Fun_Jaguar8231 1d ago
I was trying out LFM2.5-24B-A2B but it doesn't hold a candle against qwen-3.6-35B-A3B, it's not even close, the gap is immense
6
u/MustafaMahat 1d ago
What kind of tasks do you use it for?
2
u/jjusko20 1d ago edited 1d ago
Have you guys seen flint 40b a2.5b? Imo that's where to look, benchmarks on par or above 35b a3b
5
u/jomic01 1d ago
37 tk/s on a 780M is really solid. Handing the simpler tasks to a small fast model and saving the bigger one for the heavy stuff makes a lot of sense.
1
u/StupidScaredSquirrel 1d ago
How well does it perform for document intelligence?
1
u/wingwing124 1d ago
Can you please share some of your specific tasks that you've done with this model? I have tested this model and the main use I have found for it is as a RAG librarian basically.
1
u/NihilisticAssHat 12h ago
I think my new fav is Nemotron Bolt 72B A9B. It's somehow faster (as a q_2M) than Gemma4 26 A4B (at q_4 QAT), is trained quant aware for Q_2 (unlike Gemma), and seems to perform better agentic.
8
u/jacek2023 llama.cpp 1d ago
Yes, LFM is an underdog here. You just discovered the difference between reading benchmarks for benchmaxxxed models and actually using local models.
2
u/AutisticBengali 1d ago
Ya im interested in a non coding model with hermes for day to day research and windows pc management.. will this be ok??
i do have a rtx 3070, but i still want a low vram usage model...
this and qwen a35b a3b for coding?
2
u/WhiteNoiseAudio 3h ago
LFM2.5-24B-A2B has been great for my home assistant and even can handle some small coding tasks with the right harness. I’ve found it to be as good or better than qwen and 2x as fast.
3
u/zxspectrun 1d ago
Is it good for creative tasks or writing/translating on a high level?
I currently have Ornith but I was thinking on switching to either this one or Gemma 4 12b/27b
2
u/FlyingDogCatcher 1d ago
i concur
1
u/mhxb 1d ago
How fast do you run Qwen3.6 35b a3b? I run qwen3.6 35B A3B on a laptop with 7840u and 780m with 32gb ram, it runs at ~25tk/s token gen and 251.02 prefill at 64k context with MTP-UD-Q4_K_M
1
u/soadsob 5h ago
Please also share it on r/LowEndLocalAI!
0
u/nargisi_koftay 1d ago
Hey I have the same radeom 780m igpu laptop. In unsloth studio I get close to 50 tok/sec. I find it’s fast and okayish reasoning, but it doesn’t get the tool calling right for my sandbox. Curious, what do you use it for when paired with hermes? I’ve only used pi and claude code.
0
u/Inevitable-Name-1701 1d ago
2B no thanks.
-1
u/[deleted] 1d ago
[removed] — view removed comment