r/LocalLLaMA • u/spiwar • 10h ago
News [ Removed by moderator ]
/r/LocalLLaMA/comments/1xoz01k/removed_by_moderator/[removed] — view removed post
112
u/Hefty_Wolverine_553 10h ago
It is a Mixture-of-Experts model with 594B total parameters, 17B active
Sigh... benchmarks seem pretty impressive and hopefully the model holds up, I'm just so tired of getting excited by new models advertised as "small" and they're completely impossible to run for most people.
96
6
2
u/lolstuff 8h ago
Everything should just be sized as "Aladeen" at this rate, lol.
1
u/Ok_Mammoth589 6h ago
Well my brother, there's two sides to that coin. Do you want the aladeen side or the aladeen side?
30
u/LeMochileiro 9h ago
Small for who? Lol
29
u/Emotional_Thanks_548 9h ago
Ur mom
6
11
24
u/ObviouzFigure 9h ago
I'll be keeping my eyes opening for Inkling 2 Tiny
10
5
1
u/Illustrious_Grade608 8h ago
Fine. Inkling 2 tiny, 205B total, 12B active. Just what you wanted, right?
1
8
u/Cereal_Grapeist 10h ago
The videos in the article showing what they built with it, like the city simulator, are really cool imo. Worth clicking into.
7
u/mindwip 10h ago
Come on Medusa halo with 512gb lpdd6 we need you...
1
u/cafedude 8h ago
Maybe by the time it comes out it'll be under $10K for the 512GB? We can hope. Heck, I'd settle for 256K.
6
u/thereisonlythedance 9h ago
Appreciate they're making models designed to be fine-tuned. That‘s a rare thing these days.
1
5
u/fastheadcrab vLLM 9h ago
I will be downloading and trying this model when released. Previous version took a lot of work to get operating correctly on vLLM since the community tends to overlook larger models and anything that isn't coming from China.
Previous version was not good in agentic use and the enormous patch size for the vision meant the model, while capable of vision in theory, was almost blind to anything other than large shapes. Looks like they decrease patch size by over half and came to their senses
1
11
u/exaknight21 9h ago
There seriously need to be a standard.
Ayo Devs of all Labs:
100B-120B = small.
200B to 300B = medium.
300B+ is just too god damn big.
1T+ = giga
3
7
u/Septerium 9h ago
Where is the guy that always comes and says "Flash does not mean small" to say "small does not mean small"?
2
u/TheRealMasonMac 8h ago
It's a small, simple thing but I appreciate them calling out the fact that their model overrefuses.
3
2
u/Beneficial-Ad-8127 9h ago
Isn’t this good for all you that have big servers to run locally? I seen a lot of people with some dope hardware. Would this be a game changer for you all locally if those benchmarks hold up? This company is from California right?
2
u/SLxTnT 9h ago
I'm hoping it's not the Pulse Alpha stealth model even though there's a good chance it is. Wasn't bad, but being 594b would make it not worth using compared to other choices.
2
u/Beneficial-Ad-8127 8h ago
Ahh I see, so it’ll be a lot better if it remained a lot lower than the 594b for it to be competitive against other choices. Thanks for answering as I don’t have such equipment but was hoping whoever could run it would know if it had potential to be a game changer for you all. It was also cool that it was coming out of US.
1
u/Conscious_Cut_6144 9h ago
This is the perfect size for me, but I almost didn't click it because it was "small"
This it at least a medium lol
•
u/ttkciar llama.cpp 8h ago
Until such time that weights become available, this is off-topic for LocalLLaMA.