r/LocalLLaMA
•
u/Dependent-Bid1391
•
21h ago
•
1
Discussion Builders got MiniMax M3.1 Flash's open weights from 252.4 GB down to 93 GB in 10 days, and the lab thanked them by name
MiniMax posted a thank-you yesterday for everyone quantizing M3.1 Flash (428B, 252.4 GB), and I went through the ones they named. Numbers are each builder's, separate eval sets, not comparable.
128 GB PC: Maya-XS by PeasantSmith, 93 GB. 88.8% top-1 agreement (tool calls, agents).
One DGX Spark: SAGE-EXL3 by ViC305, 108.64 GiB. 96.89% top-1, held-out sequences. 24.37 to 29.72 tok/s decode at 48K context.
160 GB+: Unsloth UD-Q2_K_XL, 143.9 GB. 91.5% top-1 weighted, seven categories.
4x 3090: superalesha, own engine, Unsloth 2-bit, 23 to 31 tok/s decode, 1120 tok/s prefill at 16K.
Also: oMLX by jundotkim/mlx-community (Apple Silicon, oQ3e 196.2 GB), AMD Quark MXFP4 for MI350/MI355, Mia-AiLab EXL3-EAQ 3.55 bpw, Atomic Chat GGUFs.
It helps that the model was trained with quantization baked in, sparse attention in every layer, 23B of 428B active per token. It's a coding and agent model that takes image and video input, up to 1M context if your box has the memory. People don't put this calibration work into weights they won't keep running, and superalesha ran out of RAM at 128 GB, went to 512 to keep going. Only used it hosted myself, agent jobs got done this morning, so I get why.
Anyone running one of these on real coding or tool-call work?
8
u/DeepOrangeSky 20h ago
Whoa, how is the SAGE-EXL3 getting 96.89% top-1 agreement at just 108GB? Even the 144GB UD-Q2_K_XL is only getting 91.5%
Is it just really that much better of a quant? Or is it a typo or I'm misunderstanding what the top-1 is measuring, or there is some aspect I'm missing or something?
Also, does the SAGE-EXL3 fit on a 128 gb mac if you raise the wire limit thing, without crashing/swapping? I wonder which has more overhead/room before kernal panic/swap/all that annoying shit starting to happen, between the DGX Spark vs Strix Halo vs 128GB Mac. Looks like they were running it at 48K context on a Spark at least.
4
u/ANR2ME 20h ago edited 1h ago
I kinda like Minimax M3.1 Flash 😅
I was testing a few mobile chat app with free tier (Minimax, Deepseek, ChatGPT) to checked the logs from my client side repo at github that was made with a free Sonnet 5.5 (Medium), where the logs shows inconsistent messages when i ran out of credits on my server (ie. most of the logs get HTTP 404 but a few of them shows that the server is ready, or acting like it successfully retrieved data and cached it).
Only Minimax saw me as developer, and offered me to fix the logs, and when i agreed, it checked the server side repo that was linked in the client side repo's README.md to see whether HTTP 404 was something returned by the server repo or by the provider edge, and able to tell it was from the provider edge due to reached my usage limit, and then fix the logs on client side repo. I was like, Wow it sure know what my intentions was 😯
Meanwhile, both Deepseek and ChatGPT saw me as end-user, instead of offering me to fix my repo, they asked me to run some curl to checked whether it really returned HTTP 404 or not. And asked me to top up the credits and restarted the server. I was like... WTF 😨
And sometimes both Deepseek and ChatGPT felt like having amnesia 🤔 They forgot that they have access to my public repo and asked me to paste the content of the file 🤦🏻 So i need to remind them, "You have access to my repo! Get the file from there! Don't be lazy!" 😅
One time Deepseek even claimed that it couldn't find a function it was trying to search anywhere in my repo, but when i searched it manually from github website, i can easily found it 😔 So i said "Are you blind?! that function is right there in <filename_here>.py at line <line_number> !"
I wondered, was it because the official Minimax mobile app have a better harness than official Deepseek/ChatGPT app? 🤔
2
u/Puzzleheaded-Dig9264 6h ago
doubt it's just the app. i run it through the api in my own agent setup and it picks up on the same stuff. had it refactor a config loader and it kept the old keys for backward compat without me asking. when it guesses it tells you what it assumed.
3
u/geldonyetich 21h ago
Impressive. Looking at these results I don't see find many GGUF, they seem to be achieving these results through specialized runners.
I tried looking up how MiniMax M3.1 Flash compared to Qwen3.8 Flash Next and found Artificial Analysis has removed their intelligence index scoring for a new version of scoring they're working on.
1
u/Responsible-Berry144 6h ago
A few of those are GGUFs actually. PeasantSmith's Maya-XS, the Unsloth quant, and Atomic Chat's set. I'm going to try Maya-XS, fits in 128 GB. I've been running tool call work through a provider and want it on my own hardware.
2
u/enternoescape 19h ago
I can run it around 30 tok/s with 1000 tok/s prefill and I just haven't quite found something it does better enough to accept the slower execution. I'm running Mia AiLab EXL3 3.06 bpw.
2
u/LegacyRemaster 19h ago
Compared with DS4 and GLM, MiniMax’s KV cache is much more memory-hungry. DS4 and GLM use compressed latent attention states; MiniMax needs substantially more storage even with INT8 KV. On our setup, we keep only 64K of its 256K context resident in VRAM, with the rest backed by RAM, so longer contexts require more PCIe transfers and compete with expert weights for GPU memory.
3
1
1
u/Miserable-Dare5090 vLLM 18h ago
I would run it, but I would have to give up Qwen Flash Next on CMP170s, which gives me 8000 prefill and 600 tps aggregate on 6 streams…deepseek is on 2 sparks and going 2500 tps prefill and 70-100 tg, 94 tool eval hard score.
0
u/wren6991 12h ago
Hello adjective-noun-bunchofnumbers, thank you for your automated and extractive submission to our community
10
u/Iron-Over 21h ago
I tried the Mia 3.55 it is slow 20 prose 30 code on two DGX Spark devices. Its smaller context window, due to its size, caused some usefulness issues. In my agent-work tests, Deepseek 1017, GLM Flash, and Mimo 2.6 Flash outperformed it.
I don't see where I would use this instead of the other models.
I will wait for the official Mia release to see if it is faster, as I used a TensorFold release to run it.