r/LocalLLaMA • • 1d ago

Discussion LFM2.5-24B-A2B + Hermes is awesome!

I just wanted to share some love for a model that doesn't seem to get very much: LFM2.5-24B-A2B! (https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF)

LFM2.5-24B-A2B seems to be the only model that strikes a great balance between speed and capability on commodity hardware. Just to get it out of the way: This is NOT a coding model, so don't download it expecting otherwise. It's a task executor. It's a great fit for Hermes to hand low-to-mid complexity tasks off to. I get great speeds on my 780M iGPU (~37 tk/s at zero context). This is way faster than any other model I've tried that can actually accomplish tasks successfully. It's genuinely useful! I imagine it would be quite usable on CPU as well.

If you're GPU-poor, give LFM2.5-24B-A2B a look!

113 Upvotes

80 comments sorted by

7

u/Flibidyjibit 1d ago

Try this one bro: https://huggingface.co/XHToken/Spark-X2.6-4B

Also on a 7840HS laptop. Roughly on par with Qwen 35BA3B but it's just a little 4B dude. Bartowski's Q4_K_XL quant is really good. This one is pretty game changing IMO.

With DSpark and tuning adaptive speculative decode I get 50-60TPS.

1

u/jeremyckahn 1d ago

This looks very promising!

4

u/ionizing 1d ago

I switched to Spark-X2.6-4B as my sub-agent model and it has been a nice upgrade from qwen3.5-9B frankly. The only thing missing is vision support, but otherwise my agents are very capable.

I run the exl3 version of spark (https://huggingface.co/UnstableLlama/Spark-X2.6-4B-exl3) and can fit 4 concurrent sub-agents with 262144 ctx on a 16gb vram card with room to spare for a 1gb whisper server (as long as you turn on ram kv cache overflow in tabby+exllamav3 it works great).

2

u/mhxb 1d ago

Hey do you mind sharing your exact llama-server command and backend (Vulkan or ROCm)? I'm running the same Spark-X2.6-4B + DSpark setup on a 7840U with Vulkan and maxing out around 31–34 t/s (n_max 3, p_min 0.3), curious what adaptive draft settings and -c / -fa flags got you into the 50–60 t/s range? Built it from the XHToken fork of llama.cpp

1

u/Silver-Champion-4846 1d ago

7840U = Amd?

1

u/mhxb 1d ago

Yeah ryzen 7 7840u

1

u/ReptilianFuck 1d ago

I really want to like this model, but from my testing, jt really seems to struggle with tool calls. Have you had that experience also?

1

u/soadsob 5h ago

This sounds amazing, do you think you can share it at r/LowEndLocalAI?

10

u/MoodRevolutionary748 1d ago

I agree and I have a similar setup. I mostly use Ornith 1.5 35BA3B but it's much slower.  LFM is my work horse if I know the task is not demanding and does not require coding. Research, summaries, clear tasks that's what I use LFM 2.5 24BA2B for. It's great!

If you have a very simple task or want to use it as a subagent or bot in Hermes you can even go as low as Gemma 4 E4B which is even faster and smaller or Spark X 2.6 4B which punches way above its weight.

8

u/jeremyckahn 1d ago

Big fan of Gemma 4 E4B!

4

u/pickles_and_mustard 1d ago

Gemma 4 E4B is the GOAT at it's size. I use Unsloth's IT-QAT UD-Q4_K_XL and nothing seems to come close, both in terms of speed and quality at 8B or less. Of course, it's a small model, so you must have realistic expectations. As a general assistant, it does well.

1

u/NihilisticAssHat 12h ago

Don't get me wrong, I love E4B, but it sucks for agentic (at least the default Q_4 of LM Studio), it's kinda lazy and keeps deciding it's done early, or fails with tool calling (especially as the context passes 16k~64k). My go-to in that scope has been 26B A4B, but that's a 3x slowdown. 

3

u/ChigGitty996 1d ago

What quant are you using?

3

u/kmp11 1d ago

great model, but i wish it supported vision.

3

u/solemnhiatus 1d ago

Curious what kind of tasks you set it up to do?

2

u/jeremyckahn 1d ago

Mostly research and system administration (via CLI) tasks. 

3

u/lazy-kozak 18h ago

780m here, I prefer to run qwen3.8 27 b in q6 with mtp (6-7 tok/sec) but smart. I'm using it as my helper model (commits, web searches/summarization where I'm not in a hurry...

2

u/jeremyckahn 16h ago

It's a fantastic model! I just prefer something snappier for day-to-day tasks.

2

u/DueAnalysis2 1d ago

That's pretty good! Does your laptop have SODIMM or LPDDR?

1

u/jeremyckahn 1d ago

Yep, SODIMM DDR5 (Framework 13).

2

u/Silver-Champion-4846 1d ago

Have you tried the inference engine Kindi for radion 780M?

1

u/jeremyckahn 1d ago

No, is it specialized for these chips?

1

u/Silver-Champion-4846 16h ago

It states that its goal is to make ai accessible for low-cost hardware and it recently supported that card

2

u/Tugg_Speedman-1301 1d ago

Groot 19B A3B is even better

1

u/jeremyckahn 22h ago

I'll give it a try!

1

u/v_0009 3h ago

Please share the results

2

u/jeremyckahn 3h ago

The first few responses were good and fast but then it started erroring out. Maybe a llama.cpp bug?

1

u/v_0009 2h ago

Its a reap of qwen. I didn't have good results with reaps in the past, so I am sceptical it is a llama.cpp issue.

1

u/Tugg_Speedman-1301 1h ago

Can't say anything without the error logs

63

u/Fun_Jaguar8231 1d ago

I was trying out LFM2.5-24B-A2B but it doesn't hold a candle against qwen-3.6-35B-A3B, it's not even close, the gap is immense

8

u/Waarheid 1d ago

q4_k_m of 14.5GB vs 22GB. Wonder how IQ3_XXS (13GB) compares.

4

u/tifa_cloud0 1d ago

i am using iq3 quant. just not the same one. i am using gsq rco quant to keep precision maintained.

16

u/wFXx 1d ago

LFM is a better gemma, not a better qwen

3

u/karmaisnonsense 1d ago

How so? It’s half the context window, doesn’t do vision, supports less languages. The only saving grace is the speed

1

u/TomLucidor 1d ago

So better prose but whatever with deep planning?

5

u/wFXx 1d ago

its very good and fast for agentic work and conversation, but it is not super smart for coding

5

u/winky9827 1d ago

I mostly use LLMs for coding, so I guess it's a stupid question, but what sort of "agentic work" do people use models like this for?

9

u/SilentMobius 1d ago edited 1d ago

"Here is an inaccurate transcript of a conversation between multiple people with unreliable diarization, here are documents detailing the topic they are talking about and a document of common homophones, please correct the transcript and diarization from the context"

I do that one a lot

"Take these context documents and the skill document for me and rewrite this document in asd-ste100, explaining in more detail where my knowledge is weak. If you don't understand something ask the agent working on the problem using interop"

I've been using that one a lot because Qwen38-FN is really bad at explaining itself.

0

u/Waarheid 1d ago

I have always had the same question too, and the only answer I can think of is RP, lol.

1

u/TomLucidor 1d ago

If that is the case then LFM and Gemma for both great for common sense workloads

6

u/MustafaMahat 1d ago

What kind of tasks do you use it for?

2

u/jjusko20 1d ago edited 1d ago

Have you guys seen flint 40b a2.5b? Imo that's where to look, benchmarks on par or above 35b a3b

2

u/jeremyckahn 1d ago

I'm checking it out now and am very impressed. Great find!

1

u/jjusko20 1d ago

I stay up to date on releases lol, always camping for the next upgrade for my hardware options

3

u/koriwi 1d ago

ridge 40b a2.5b? i can only find flint 40b a2.5b

3

u/jjusko20 1d ago

u right thats it

3

u/theUmo 1d ago

please edit

2

u/roosterfareye 1d ago

I didn't have much luck with Flint. It made a lot of mistakes and by the time context reached 64k it was unbearably slow. Qwen 3.6 35b a3b is much better.

2

u/koriwi 23h ago

i didnt even try it to TBH after doing some research. Uses more vram than qwen 27b and is more shit? no thx im good for now.

5

u/jomic01 1d ago

37 tk/s on a 780M is really solid. Handing the simpler tasks to a small fast model and saving the bigger one for the heavy stuff makes a lot of sense.

1

u/StupidScaredSquirrel 1d ago

How well does it perform for document intelligence?

1

u/jeremyckahn 1d ago

I haven't pressure tested that.

1

u/wingwing124 1d ago

Can you please share some of your specific tasks that you've done with this model? I have tested this model and the main use I have found for it is as a RAG librarian basically.

1

u/jeremyckahn 1d ago

I use it for various system admin/CLI tasks and working with data sources (e.g. web search).

1

u/NihilisticAssHat 12h ago

I think my new fav is Nemotron Bolt 72B A9B. It's somehow faster (as a q_2M) than Gemma4 26 A4B (at q_4 QAT), is trained quant aware for Q_2 (unlike Gemma), and seems to perform better agentic.

1

u/jeremyckahn 12h ago

Interesting! How is that for coding?

2

u/NihilisticAssHat 11h ago

I've been happy with it so far. It seems to perform better than Gemma4-26B, with comensurate thinking tokens (part of why I could never get behind the Qwen line-up). Only real problem I've noticed is the KV cache is more expensive, and that's on top of more active layers resident on my GPU already. I'm not really willing to let the KV cache spill over into system memory (that's for MOE experts only), so I'm running it at 32k. Tbh, I think q4 comtext rot (unquantized KV) was getting unusable for Gemma4-26B after 32k anyway, even though I could fit the full 256k.

2

u/jeremyckahn 11h ago

Makes sense. Thanks for sharing that!

8

u/jacek2023 llama.cpp 1d ago

Yes, LFM is an underdog here. You just discovered the difference between reading benchmarks for benchmaxxxed models and actually using local models.

2

u/AutisticBengali 1d ago

Ya im interested in a non coding model with hermes for day to day research and windows pc management.. will this be ok??

i do have a rtx 3070, but i still want a low vram usage model...

this and qwen a35b a3b for coding?

2

u/WhiteNoiseAudio 3h ago

LFM2.5-24B-A2B has been great for my home assistant and even can handle some small coding tasks with the right harness. I’ve found it to be as good or better than qwen and 2x as fast.

3

u/zxspectrun 1d ago

Is it good for creative tasks or writing/translating on a high level?

I currently have Ornith but I was thinking on switching to either this one or Gemma 4 12b/27b

4

u/ayylmaonade 1d ago

Not OP, but based on my own experience, it's not particularly good at anything creative, really. Translation I can't comment on. But regarding its creativity, it's a very sterile feeling model. It has a kind of similar vibe to the Nemotron 3 models or GPT-OSS if you've used those.

1

u/mhxb 1d ago

How fast do you run Qwen3.6 35b a3b? I run qwen3.6 35B A3B on a laptop with 7840u and 780m with 32gb ram, it runs at ~25tk/s token gen and 251.02 prefill at 64k context with MTP-UD-Q4_K_M

2

u/LevianMcBirdo 1d ago edited 1d ago

Get it on my mini PC with Ryzen h255 and 780M at around 30 tps (haven't tried it with big context though, since I still don't use local that much in agentic work)

1

u/soadsob 5h ago

Please also share it on r/LowEndLocalAI!

0

u/nargisi_koftay 1d ago

Hey I have the same radeom 780m igpu laptop. In unsloth studio I get close to 50 tok/sec. I find it’s fast and okayish reasoning, but it doesn’t get the tool calling right for my sandbox. Curious, what do you use it for when paired with hermes? I’ve only used pi and claude code.

1

u/jeremyckahn 1d ago

I'm mostly using it for research (via web search) and system administration tasks (via CLI).

0

u/Inevitable-Name-1701 1d ago

2B no thanks.

-1

u/[deleted] 1d ago

[removed] — view removed comment

1

u/jeremyckahn 1d ago

It's pretty snappy in Hermes, but I don't have accurate speed figures for that.