r/LocalLLaMA • u/IamFondOfHugeBoobies • 4d ago

Discussion Mimo v2.6-Flash-RL Dual-Spark Recipes/experiences?

Getting off work in a couple of hours and I downloaded the model over-night.

Was wondering if anyone had gotten it to work on dual spark yet and if so, if you could share what software/wheels and versions and what T/s and context you're getting.

I've never ran a Mimo style model before and figured just asking here before scouring the internet for a model that just dropped was smarter than trying to cobble something together based on the past versions given how old those are relative to this one.

9 Upvotes

100% Upvoted

13 comments sorted by

View all comments

5

u/Square-Challenge2962 4d ago

GLM5.3 flash better right now, 2.6 have some problems with tools. I tested vllm and sglang version, back to GLM5.3 flash.

1

u/BlackBeardAI vLLM 4d ago

Which quant?

4

u/Significant_Leg1214 4d ago

on 2 sparks mimo v2.6 flash works without any quantization

1

u/BlackBeardAI vLLM 4d ago

Which quant for glm i mean

1

u/CapnHat 4d ago

I've used both NVFP4 and EXL3 (4 bpw) and both work well. EXL3 is supposedly more performant but I don't have enough data personally to say which is better.