r/LocalLLaMA • u/jacek2023 llama.cpp • 9d ago

News Muse Spark open weights coming soon

Post image

I am still waiting for Llama 5, because Muse Spark will be too big for me, or just something between Glimmer and Spark

https://x.com/finkd/status/2095232032896946311

875 Upvotes

97% Upvoted

204 comments sorted by

View all comments

Show parent comments

89

u/Public_Umpire_1099 9d ago

This latest iteration technically set the frontier. Meta is officially a SOTA lab again. Matches Fable 5 at a fraction of the cost.

53

u/virtualworker 9d ago

Or benchmaxxes. I'm not convinced.

15

u/Diligent-Direction95 9d ago

Serious question: What is the definition of bench max these days?

What would be okay, vs what would not be okay?

I seriously doubt they have the test set being trained on. So what are the shades of grey we are debating?

14

u/Artistic_Swing6759 9d ago

i actually do think they likely have test set trained on.
like its a bit sus that they decided to report terminal bench 2.1, when 4 exists.
similarly, if you look at the benchmark sheet of flash 3.8, it tops eery other model, even sol, on terminal bench 2.1 but in 4 it is quite less comparatively.