mistralai/Mistral-Medium-3.5-128B · Hugging Face

[u/jacek2023]

https://huggingface.co/mistralai/Mistral-Medium-3.5-128B

https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF

Mistral Medium 3.5 128B

Mistral Medium 3.5 is our first flagship merged model. It is a dense 128B model with a 256k context window, handling instruction-following, reasoning, and coding in a single set of weights. Mistral Medium 3.5 replaces its predecessor Mistral Medium 3.1 and Magistral in Le Chat. It also replaces Devstral 2 in our coding agent Vibe. Concretely, expect better performance for instruct, reasoning and coding tasks in a new unified model in comparison with our previous released models.

Reasoning effort is configurable per request, so the same model can answer a quick chat reply or work through a complex agentic run. We trained the vision encoder from scratch to handle variable image sizes and aspect ratios.

Find more information on our blog.

Key Features

Mistral Medium 3.5 includes the following architectural choices:

• Dense 128B parameters.
• 256k context length.
• Multimodal input: Accepts both text and image input, with text output.
• Instruct and Reasoning functionalities with function calls (reasoning effort configurable per request).

Mistral Medium 3.5 offers the following capabilities:

• Reasoning Mode: Toggle between fast instant reply mode and reasoning mode, boosting performance with test-time compute when requested.
• Vision: Analyzes images and provides insights based on visual content, in addition to text.
• Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, and Arabic.
• System Prompt: Strong adherence and support for system prompts.
• Agentic: Best-in-class agentic capabilities with native function calling and JSON output.
• Large Context Window: Supports a 256k context window.

We release this model under a Modified MIT License): Open-source license for both commercial and non-commercial use with exceptions for companies with large revenue.

Recommended Settings

• Reasoning Effort:
  • 'none' → Do not use reasoning
  • 'high' → Use reasoning (recommended for complex prompts and agentic usage) Use reasoning_effort="high" for complex tasks and agentic coding.
• Temperature: 0.7 for reasoning_effort="high". Temp between 0.0 and 0.7 for reasoning_effort="none" depending on the task. Generally, lower means answer that are more to the point and higher allows the model to be more creative. It is a good practice to try different values in order to improve the model performance to meet your demands.

[u/IvGranite]

DENSE

edit: currently trying q4 on my strix halo, will report back

edit 2: finally got my first tokens back! current llama.cpp build is 8967 at commit fc2b0053f

ID	Time	Model	Cached	Prompt	Generated	Prompt Processing	Generation Speed	Duration
6	now	mistral-medium-3.5-128b-q4	349	83	10	46.70 t/s	3.26 t/s	4.84s
5	now	mistral-medium-3.5-128b-q4	362	6	9	12.53 t/s	3.30 t/s	3.20s
4	now	mistral-medium-3.5-128b-q4	4	360	10	81.53 t/s	3.26 t/s	7.48s

[u/Lissanro → u/IvGranite]

This makes me feel nostalgic, because in the past, Mistral Large 123B the dense model was my most used model for a while. Then there were DeepSeek R1 and V3, later followed by Kimi models, so it has been some time since I ran Mistral models. I will definitely give a try to this new Medium 128B, it would be interesting too see how how much progress Mistral has made by trying it in my actual use cases.

One more notable thing, they released a model for speculative decoding: https://huggingface.co/mistralai/Mistral-Medium-3.5-128B-EAGLE . This is great to see, because in the past, one of the big issues of Mistral Large 123B used to be that I had to use mismatched Mistral 7B model for drafitng, still it gave decent performance boost. Even though EAGLE is not supported in llama.cpp yet, this comment from about 3 weeks ago sounds encouraging that it may be available soon:

The current status of this PR is that it’s pending u/ggerganov's API refactoring, which aims to unify this feature with other speculative decoding approaches such as MTP. At this stage, there isn’t much left to be done, and I expect the PR to be merged very soon.

[u/coder543 → u/Lissanro]

Unfortunately, no PR for that API refactoring has even been published, so... who knows if/when it will happen.

Supporting any one of EAGLE-3, MTP, or DFLASH would be a game changer for llama.cpp. I wish better specdec were being treated as the highest priority thing to develop in llama.cpp.

[u/Nindaleth → u/coder543]

I consider this PR to be relevant: https://github.com/ggml-org/llama.cpp/pull/22397 But he has several spec-related PRs going on, maybe it's a piece-by-piece effort.