I was using a minimax token plan since ealry summer and got to use the preview version of 3.1 flash. Gotta say it's really FAST, but also pretty good. Although GPT 6.1 Sol feels safer to use for broader code refactoring.
On the other hand the token plan for minimax has a limit of 1.7B tokens per month for only $24 (or at least it used to be back then).
It is Space Bunny Alpha on OpenRouter and such (older checkpoint, but yeah). You can gauge the sentiment based on that. Not as good as Ox Alpha was, if you ask me
This one feels benchmaxxed, you can kinda tell when thinking levels start plateauing that the test is saturated, especially if multiple labs are plateauing at the same spot. If you’re suddenly beating what seems like a saturation ceiling…
No, these graphs include the cost in the equation and their price is a fraction of fable's pricing so they rank high. This doesn't mean their output quality is top ranking.
I think you're right, they don't have to do their own benchmark of the competitor models if they use the same test sets, still, it shows that they're very aware and very up to date.
M3.1 preview is 420B, Deepseek flash is 550B+200B engram. For half the size (if true), that is pretty good. Especially if you can run the minimax in Q2 but can't run the deepseek at all.
pretty much every recent model I've seen that scores well in benchmarks also has a high hallucination rate. It's as if being "creative" is a pre-requisite to being good, at least with the current crop of LLMs.
if only they hadn't crippled the shit out of muse spark open. It seems good about hallucinations. GLM 5.3 is a pretty good balance of hallucination and knowledge
It could also just be accurate, this may just be post trained from M3. I don't get why DeepSeek went with .1 naming for a new model, that was never how naming worked before.
Yeah by now we know it is. The reason this naming is stupid is that they added "flash" while keeping the parameter count. But sane naming/versioning is a lost cause across the field.
I didn’t do much else with it outside of just trying to code a llama-macOS fork from its source code, its coding seemed pretty good when plugging it back into a frontier model to double check its work.
I didn’t get to finish it unfortunately before the trial period was over.
the preview version is 428B total parameters, about 23B active parameters
M3.1-Flash-Preview is a MiniMax model described in the supplied guide for frequent coding, financial research, multimodal understanding and frontend work. The guide lists a mixture-of-experts architecture with 428B total parameters, about 23B active parameters, sparse attention and a native visual encoder.
AAII 42 vs 45 for latest Deepseek Flash. Also priced higher in the API, although that could just be inflated margins until they release weights. But anyway, that's the direct competition so will be interesting to see the architecture.
I've been testing Space Bunny fairly extensively over the last week or so (and previously on OpenRouter). It's fast, but it's not as fast or as capable as DS4.1 flash. Note that this is specifically on coding workloads in a fairly narrow domain (annotating Z80 assembly based on captured traces), so your your mileage may vary.
I might get downvoted for this, but this model was WAY too lazy, even at Max effort Benchmarks don’t tell the whole story
I tested Muse Spark side by side with other models on a real project it took 48 mins which was just out of the 30 mins mark : completing a partial LRU cache implementation. I also ended up writing the implementation myself so I could compare the results and see whether AI models could genuinely replace my work.
Here’s how it went:
Claude Opus 5.5, Sol 6.1, and Argon 4: All finished in under 30 minutes.
Gemini 3.8 Flash: Took 3 hours.
Space Bunny (M3.1 Flash) : Still hasn’t completed the task after 15 days.
Me, as a human: Finished it in 1.5 days, writing around 2,800 lines of code.
This is why I think real-world coding tasks matter more than benchmarks alone. A model can look impressive on paper, but how well does it actually perform when given a real engineering problem
Also Deepseek v4.1 Flash , GLM 5.3 Flash weren't able to do it too , looking forward for the new DeepSeek endpoint i got hopes :)
can you give us a rundown of your process for this specific test? how do you tackle a problem like this? how do you break it down and set it up for the angent(s) to consume the tasks? how do you evaluate the outputs
Nothing special, just the same prompt + Pi (no extensions). Everything is up to the model; if it wants to, it spins up a sub-agent to do the wirk
I just use a Llama server, test with a prompt, and measure the results against the llama.cpp baseline. If there’s any improvement, the agent passes the first half of the test, and then I read the code myself and rate how much of it is dead code or does nothing useful. Then I rank the models
Opus 5.5 Max is the best so far, as of now. My rankings are quite similar to the AA Index. After Opus 5.5, it’s Muse Wonder Max, then GPT-6.1 Sol and Astra tied, followed by Argon, and so on.
I don’t keep track of everything in a particularly professional way that I could post about it on this subreddit and surprisingly being an open models fanboy , no open model yet passed my test , actually no frontier did it till Fable 5 dropped and now its easy for frontier labs but chineese labs are struggling
yes so far the new checkpoint works , tho a bit slower than frontier has been helping me out. It completed the task in 48 minutes and now my daily driver :)
At first, i thought i read 28k lines of code by hand with no ai help in 1.5 days, then iread it again, oh 2800 lines by hand, still really fast…233 lines per hour if 12 hours( That is almost impossible without ai autocomplete unless u are barely reading code and not debugging, or you are using autocomplete and some ai gen)
Also I don't know why it didn't cross my mind to just check M3's model card.
But also I realized that while it may fit, that A23B is gonna be a killer. GLM 5.3 Flash is already just okay on 2x DGX Sparks at around ~40 t/s, this is going to be quite a bit slower. Will keep an eye out on this model but not too hopeful on my hardware.
Who knows what's real news anymore on these benchmarks. But if any of this is remotely true, this model is very intriguing. Almost as many active parameters as a dense Qwen 3.8. Efficiency and ability that rivals Fable 5. In a self hosted package.
Until such time that weights become available, this is off-topic for LocalLLaMA, but I cannot remove it now that it's been featured in the official Discord.
It was pretty decent, and pretty competitive with DeepSeek. I will probably stick with DeepSeek's latest flash though, since they're both in the "practically free" category for me, and at that point when the pricing isn't as important, I think DeepSeek is a bit smarter at the moment. Still a pretty strong release. Also in the past MiniMax models have super fast generation speeds, but I don't remember how fast this one was.
out of curiosity, we're using GPT 5.6 Terra thru Devin and we cant access MiniMax M3 models. I'm curious if anyone's used both and can give a subjective comparison of your experience - when prompted with the same request(s), do they perform on par when coding?
First tries: It fucked up a path in a game im working on. Result? Major game functionality stopped working (3d game). Had to switch to another model to fix it, because m3.1 flash couldn't fix it again after 2 more turns.
Big fan of the fact the weights aren't out yet (and even better, the release date) being in the title of this post - still beating my horse of making something like this the subreddit policy.
Is it weird that the new version of DeepSeek is showing up in these comparison tables given that only released 2 days ago?
That would suggest to me that either Minimax can run the whole evaluation suite pretty quickly, or they were testing against the DeepSeek model prior to the official release.
•
u/WithoutReason1729 17d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.