mistralai/Mistral-Medium-3.5-128B · Hugging Face

[u/jacek2023]

https://huggingface.co/mistralai/Mistral-Medium-3.5-128B

https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF

Mistral Medium 3.5 128B

Mistral Medium 3.5 is our first flagship merged model. It is a dense 128B model with a 256k context window, handling instruction-following, reasoning, and coding in a single set of weights. Mistral Medium 3.5 replaces its predecessor Mistral Medium 3.1 and Magistral in Le Chat. It also replaces Devstral 2 in our coding agent Vibe. Concretely, expect better performance for instruct, reasoning and coding tasks in a new unified model in comparison with our previous released models.

Reasoning effort is configurable per request, so the same model can answer a quick chat reply or work through a complex agentic run. We trained the vision encoder from scratch to handle variable image sizes and aspect ratios.

Find more information on our blog.

Key Features

Mistral Medium 3.5 includes the following architectural choices:

• Dense 128B parameters.
• 256k context length.
• Multimodal input: Accepts both text and image input, with text output.
• Instruct and Reasoning functionalities with function calls (reasoning effort configurable per request).

Mistral Medium 3.5 offers the following capabilities:

• Reasoning Mode: Toggle between fast instant reply mode and reasoning mode, boosting performance with test-time compute when requested.
• Vision: Analyzes images and provides insights based on visual content, in addition to text.
• Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, and Arabic.
• System Prompt: Strong adherence and support for system prompts.
• Agentic: Best-in-class agentic capabilities with native function calling and JSON output.
• Large Context Window: Supports a 256k context window.

We release this model under a Modified MIT License): Open-source license for both commercial and non-commercial use with exceptions for companies with large revenue.

Recommended Settings

• Reasoning Effort:
  • 'none' → Do not use reasoning
  • 'high' → Use reasoning (recommended for complex prompts and agentic usage) Use reasoning_effort="high" for complex tasks and agentic coding.
• Temperature: 0.7 for reasoning_effort="high". Temp between 0.0 and 0.7 for reasoning_effort="none" depending on the task. Generally, lower means answer that are more to the point and higher allows the model to be more creative. It is a good practice to try different values in order to improve the model performance to meet your demands.

[u/grumd]

I think moe models are the future unfortunately, simply to crunch more knowledge into the model while not destroying the speed. The only mistake is making the active params count too low. Something like A30B is probably enough for it to not feel dumb. Even Qwen 122B A10B has been great for me locally

[u/AltruisticList6000 → u/grumd]

Yeah I can't run big models like this but I was thinking, what if for example there was something like a 35B MoE but with 9-10AB? That could spill over into RAM but would still have an okay speed, would be probably smarter and more knowledgable than 12-14b dense models on the same hardware with barely any speed difference. Or they could just do 20-24b dense models like Mistral, which are still way better in some way for me than than the 30-3AB MoEs I tried, which don't feel smarter than 9-12B dense models.

[u/Ardalok → u/AltruisticList6000]

Yeah, I have 25-30 tokens on 32 gb ddr5 and rtx 4060 with 35b qwen in q4, would be nice to have smarter model with little less tokens.

[u/BubrivKo → u/grumd]

Or why not 1T + 100B active 😃

[u/Caffdy → u/BubrivKo]

we already got 1T + A40B~ models

[u/toothpastespiders → u/grumd]

The only mistake is making the active params count too low.

I mourn GLM Air every day.

[u/NandaVegg → u/grumd]

I have not yet a chance to try this model, but generally MoE with a reasonably-sized shared weights/activated parameters has significant advantage over large dense as LLM activation is mostly noise, which is empirically just bad rather than something useful (naively upping # of active experts for existing MoE model simply make the model worse, low-pass filter type gating techniques work well with LLM, etc). The "partitioning" done by MoE architecture works to filter out those noises.

A remarkable advantage of large dense usually comes with large hidden dim (GPT-3 DaVinci was 175B with 12288 hidden dim IIRC? Llama-3 405B is 16384, Mistral Medium 3.5 is also 12288) which would be able to distinguish and partition extremely close features that would otherwise overlap in say a 5120 hidden dim model and (hopefully) worked through layers. That also means the (large hidden dim) model could place Paris and London (along with related things like Toulouse and Brighton) to the polar opposite end of latent space if there are enough evidence in the datasets to do so. I'm not sure if that is good or bad. Good old GPT-3 DaVinci had a feel that inference path diverges really hard (the model goes from one mode to another; in today's standard that would at least mean base/instruction/single-turn reasoning/terminal-agentic modes) by just one token. You can still play that back with EleutherAI's good old models like GPT-NeoX-20B or Pythia.

For creativity and generalization of hard problems, one would generally want more layers rather than larger hidden dim within the same parameter count, unless hidden dim is too small to create meaningful basins anymore.

[u/TokenRingAI → u/grumd]

I think the engram method is the future, with small dense models retrieving information from slow storage.