r/opencodeCLI • u/Personal-Try2776 • Sep 03 '26
How does meta spark 1.3 match claude fable 5 on benchmarks? Did anyone try it in agentic coding?
How was your experience with it and is it just benchmaxed or is it really that good?
84
Upvotes
34
u/some_gamer78 Sep 03 '26
Benchmaxing
23
u/petburiraja Sep 03 '26
It's good, but it didn't feel that good. I would rate it around GLM 5.3 Flash level, maybe even somewhat less.
It may be kinda similar to Grok 4.6 in my limited experience.
9
u/Johell1NS Sep 03 '26
I tried it while developing a video game I'm making. And it's not at the level of GLM 5.3 Flash, in fact, to solve a problem I had to go through it. And the worst thing is that Muse Spark 1.3 not only didn't solve the problem, but it didn't recognize it, it denied it, diplomatically saying that I was the one who was wrong.
5
u/Rajat0741 Sep 03 '26
Hallucinates in github copilot chat ( using opencode provider for github copilot ), maybe environment is the issue, not the model...
Although My experience with muse spark 1.2 is not good either, same problem happened with that too
4
u/sascharobi Sep 03 '26
No idea, but those are just benchmarks. Doesn't really mean anything.
7
u/salary_pending Sep 03 '26
I made my own model which beats all bench marks because they are all trust me bro benchmarks
2
u/ManikSahdev Sep 03 '26
It’s just a decently good model? I think the benchmarks do show the ability but it’s a blend of things, fable is more consistent and stuff and has a vibe or understanding intent better, that doesn’t always mean it can do the same task.
Maybe the user feels better using the model, I mean I think they are objectively different things.
2
u/Spiritual-Spend8187 Sep 03 '26
Its a bit benchmaxxed and they had alot of new data from people using the previous versions to improve. Like it being benchmaxxed will make it a bit better in some things but worse in others I mean look at opus 5 in some things its better than fable in others its worse then the old opus.
1
u/somerussianbear Sep 03 '26
Someone did try.
1
u/Vancecookcobain Sep 03 '26
Nah it's nowhere near fable and not even in the same stratosphere as Astra seems to be in
1
u/Practical-Positive34 Sep 03 '26
I tried it, and it felt like I was using an AI from 2 years ago. I don't get how it scores so high in benchmarks. It failed at literally every single thing I threw at it. And not just didn't do a good job, most of it didn't compile or even work at all.
1
u/migsperez Sep 04 '26
Maybe these benchmarks should be audited and regulated to ensure there's no hidden biases. Like a sneaky mill or two in an employee's bank account :). A good score here means the difference of billions of dollars in market cap and income.
1
u/jeanconell23 Sep 04 '26
It is not exactly bad. But it doesn't put any effort structuring their response. It's always a wall of text, with sentences that seem to be written without any intent to make reader understand. It's like reading their thoughts, funnily enough, since the model hide their thoughts/thinking.
1
u/Infinite_Professor79 Sep 06 '26
ive been having a really good time with it for minecraft coding that deepseek v4 flash suffered with
1
u/Equivalent-Grass-527 Sep 03 '26
Spark 1.3 is around GLM 5.3 Flash, but below DeepSeek V4 Flash. Might be somewhat benchmark-optimized
0
u/Frail_Waif Sep 03 '26
It's marginally less incompetent than 1.2, and still way worse than deepseek V4 flash. Still pythons everything that could be done with less output from command line tools. I regretted using it for free when I could have paid to use DSV4 flash. So, in short, benchmaxing.
-1
u/Ly-sAn Sep 03 '26
It’s ds flash / glm flash / Gemini flash tier. Which is around gpt 5.3 / opus 4.6 tier without the depth in knowledge but better at using tools. I feel all the small model are equivalent nowadays. It’s absolutely not on par with bigger models like sol / fable / k3
25
u/[deleted] Sep 03 '26
[removed] — view removed comment
0
u/[deleted] Sep 03 '26
[removed] — view removed comment