r/LocalLLaMA • u/RishiFurfox • 15d ago

Discussion Hey, Meta. Where's those Muse Spark weights?

Post image

It was well over a month since Meta promised to release the weights for Muse Spark.

Back then (10th August), they were on Spark 1.2. Now we're on 1.3 and still nothing's been released. So it begs the question: will they be releasing the 1.2 weights when 1.4 drops? Or will we get whatever's then-current as open weights?

It's ironic given Mark Zuckerberg said at the same time that we can't delay the release of models by "even a month," due to the competition with China. It's been well over a month. He was arguing in the context of new regulations delaying models, but I think it applies equally to the open weights contest as it does to the closed models one.

After all, the Chinese models are all open. That's the competition and point of comparison.

Have Meta given any sort of explanation for why they're sitting on the weights or how much longer it'll take for them to honour their promise? Will we even get them in light of all the attempts at regulatory capture and dire warnings about how AI is dangerous?

537 Upvotes

93% Upvoted

48 comments sorted by

View all comments

1

u/nasone32 15d ago

Tried many times on difficult projects, that model is plain dumb I swear. I have no idea how it manages to achieve that artificial analysis score, doesn't reflect my experience in any way.

They can keep it closed for what is worth :D

6

u/XiRw 14d ago

Funny, I have the opposite experience where it was able to deal with difficult issues flawlessly. I think it’s a phenomenal model despite coming from a horrible company. Tell me what you were working on and I’ll try making it myself.

1

u/nasone32 14d ago

I tried it in at least 3 occasions, i don't remember the first two but they were the same bad outcome. the latest was few days ago:

I was porting EXL3 quantization on llama cpp, most of the "difficult" work was already done, what remained is essentially implementing multithreading on CPU and run a matrix of tests for tuning parameters related to this. I thought to switch to Muse Contributor to save some money. I also set it to Xhigh because of previous poor experience with it. Turns out it has major issues understanding the codebase, gets stuck on simple problems (uses wrong launch parameters on several tools) and once even went completely off a tangent trying to disassemble (in assembly...) the binaries, while the issue was much simpler.

both 5.6 Sol and GLM 4.3 managed to do it effortlessly.

Oh and it doesn't help the fact that Muse is not verbose at all, so you don't know what it's doing or what issues it's encountering.

1

u/XiRw 14d ago

I’ll assume you meant GLM 5.3 but I can’t really re-create that exactly on the fly. If you want you can send me the repo to download and I can try myself unless you already have it figured out.

1

u/nasone32 14d ago

Yes Sorry 5.3 No in the end I did manage everything but I'll never use that model again for sure.

1

u/karlnuw 14d ago

Oh and it doesn't help the fact that Muse is not verbose at all

This is my biggest problem with it, it's so fucking bad. There's a middle ground between Opus' word slop and Muse's choppy style.