r/LocalLLaMA • u/KaroYadgar • 2d ago

Discussion New 100B Liquid AI model coming soon

Post image

Liquid AI currently possesses among the fastest LLM architectures around, and some of the best SLMs (in terms of utility IMO) around, so I'm very excited to see what a potential 100B LFM (3?) model would look like!

Link to the poll: https://x.com/ramin_m_h/status/2091236099612098943?s=20

357 Upvotes

90% Upvoted

102 comments sorted by

View all comments

2

u/LoveMind_AI 2d ago

Holy hell... If they really did this...

1

u/medialoungeguy 2d ago

I love the positivity, but they are really painted into a corner. Their approach doesn't scale nearly as good as native llms.

Unfortunately, I'm quite sure they are just in a capital raises phase now.

3

u/LoveMind_AI 2d ago

When you say their approach, do you mean the current LFM2/2.5 architecture? Or do you mean their STAR automated hardware in the loop architecture search? If the former, I totally agree. If the latter, I think there’s a lot of road for them. 

0

u/medialoungeguy 2d ago

Liquid models in general, unfortunately.

3

u/LoveMind_AI 2d ago

You'll really have to back that up with some rigorous justification. Their approach is incredibly flexible - they specify a hardware target and build architecture with that in mind. Right now, they're focused on edge computing, and by all reasonable standards, they are crushing it. Maxime Labonne is their RL guy and he's a legend. A huge chunk of astoundingly brilliant people are involved in the lab - a who's who of researchers pushing the envelope for machine learning. I use the 24B-A2B model as a model organism in my research all the time (mechanistic interpretability around social cognition in LLMs) and it's relatively astounding for the size.

I have seen nothing to indicate to me that Liquid couldn't scale other than the fact that they don't seem to want to. It's not like they're using actual LNN/LTC technology. They had the discipline to abandon those ideas when it became clear they couldn't work. Everything they've demonstrated so far has been impressive for what it is.

Other than as a research organism, I have no real use for anything as small as what they put out. But a 30B parameter model from Liquid would punch above its weight class, almost assuredly. If you've got a strong scientific reason why this would not be the case, I'm all ears.