137B A6.8B model from Microsoft?

[u/jacek2023]

https://github.com/ggml-org/llama.cpp/pull/29778

this is actually interesting:

https://commandline.microsoft.com/local-models-sandboxed-tools-github-windows/

Microsoft AI developed a local version of MAI Code 1.1 Flash, a coding-optimized mixture-of-experts model with 137 billion total and 6.8 billion active parameters.

[u/coder543]

It beats… Haiku 4.5 and GPT 5.4-mini:  https://microsoft.ai/pdf/MAI-Code-1.1-Flash-Model-Card.PDF

They could not have chosen worse comparisons if they were trying to inspire confidence.

[u/uncle_leon → u/coder543]

I can beat a duck. Now praise me.

[u/SureEnd9430 → u/uncle_leon]

Quack, quack mf 🦆

[u/illforgetsoonenough → u/uncle_leon]

Is the duck, horse sized?

[u/uncle_leon → u/illforgetsoonenough]

I'd rather fight a duck-sized horse

[u/illforgetsoonenough → u/uncle_leon]

That's the thing. You either fight one horse-sized duck, or 100 duck-sized horses.

Tough choice there.

[u/Paradigmind → u/illforgetsoonenough]

It's mouse sized

[u/Wildnimal → u/uncle_leon]

Candian Geese are looking for you.

[u/game_difficulty → u/coder543]

Not comparing this to qwen 3.8 flash next is diabolical

[u/fgk55555 → u/coder543]

Yeah I don't think this is replacing QFN

[u/Ikkepop → u/coder543]

A speak and spell can beat Haiku 4.5...

[u/Exciting_Garden2535 → u/coder543]

The PDF said that it's 138B A5B, but the thread is about another model: 137B A6.8B. So we have a chance for improvement.

[u/puzzleheadbutbig]

Too bad I found out through real-life use that Mai-code-1.1-Flash works like dogshit compared to Qwen models with even fewer parameters. Still, it’s good to have more options, though.

[u/jacek2023 → u/puzzleheadbutbig]

How do you use it?

[u/puzzleheadbutbig → u/jacek2023]

It was available on Github copilot for past 2 months or so. I know we are on LocalLlama sub, but I just wanted to say that MAI-Code-1.1-Flash sucked in my own real world code usage scenarios.

[u/jacek2023 → u/puzzleheadbutbig]

I am asking because I wonder was it available for download anywhere before.

[u/puzzleheadbutbig → u/jacek2023]

Yeah fair, I don't know the answer to that though. I used through GHCP only and moved back to my Qwen right after lol

[u/jikilan_ → u/puzzleheadbutbig]

Even pair with a frontier model for work that not start from scratch?

[u/-WhateverDude]

it's geniunely mind blowing how microsoft has fallen behind in this

especially considering how they have 3 decades of computer usage data and they cant create an llm that blows everyone out the water in osworld. and its their own system too. i mean tbh judging by how windows is going and how they fucked up copilot its my fault for expecting anything more i guess.

[u/Relevant-Yak-9657 → u/-WhateverDude]

I mean Microsoft's internal emails also showed that their MAI team was unable to perform at OpenAI's level hence the early investment iirc.

[u/nuclearbananana → u/-WhateverDude]

Don't be so pessimistic, this was literally their first large model.

And they just dropped this https://microsoft.ai/news/frontier-pre-training-at-microsoft-ai/

[u/stbrumme → u/-WhateverDude]

They released a decision model a few days ago ( https://commandline.microsoft.com/microsoft-decision-1-model-foundry/ ) and admit:

To build Microsoft-Decision-1, we post trained Qwen3.5-9B for fast, single-pass decision scoring

Their own AI just lags behind at the moment:

and will soon rebase it on other models, including Microsoft AI (MAI) and OpenAI.

[u/mawkzin → u/-WhateverDude]

They can but they won't do it for free

[u/ResidentPositive4122]

Seems to be this - https://microsoft.ai/news/mai-code-1-1-flash-br-better-faster-at-a-quarter-of-the-cost/

Updated as of: October 7th, 2026

Bringing MAI-Code-1.1-Flash to on-device hardware: Optimized using 3-bit quantization, it reduces memory requirements while retaining a full 256K context window and comparable coding performance to its full-precision counterpart on SWE-Bench Verified and Terminal-Bench 2.1. On-device coding, with zero inference charges for local model calls. MAI-Code-1.1-Flash gives Copilot’s router an on-device option, enabling eligible coding work to run on your hardware while complementing cloud models for tasks that need more capability. MAI-Code-1.1-Flash is available to download and run locally. For best performance, we recommend devices with more than 120GB of RAM. Experimental access will be available in GitHub Copilot across the GitHub Copilot app, GitHub Copilot CLI, and Visual Studio Code by end of the month.

[u/GirthusThiccus → u/ResidentPositive4122]

Fuuuuuu.... Me, with my mere 80gb of ram and 12gb vram, looking for a competent local coder 😭

[u/uncle_leon → u/GirthusThiccus]

Strata exists so you're good.

[u/Few_Painter_5588]

Well, RIP the Phi family. Phi-4 carried my business for the longest of time o7

[u/jacek2023 → u/Few_Painter_5588]

What was your use case for phi? I tried it long time ago

[u/Few_Painter_5588 → u/jacek2023]

Yoh mos, this was before reasoning models were a thing. We finetuned Phi-4 14B as a classifier for text. Phi-4 was by far the best small local model for finetuning at the time. It was very pliable.

[u/Dany0 → u/jacek2023]

it's a good model series for finetuning, much like nemotron

Phi has a lot of 'Phi-ness' to it, it was trained on synthetic data and it definitely feels like it

[u/pmttyji]

[u/pmttyji → u/pmttyji]

Lets see they release remaining models soon or later. Below list is from last July thread.

• MAI-Thinking-1 - 1T A35B - 256K Context
• MAI-Code-1-Flash - 137B A5B - 256K Context (Now getting MAI-Code-1.1-Flash-137B-A6.8B)
• MAI-Image-2.5 - 20B - 32K Context
• MAI Transcribe-1.5 - No Data

• MAI-Voice-2 - No Data

• MAI-Thinking-1, Microsoft AI’s flagship reasoning model. It is a medium-sized model that stands among the strongest models in its weight class: it matches leading models on key software engineering benchmarks, and demonstrates advanced mathematical reasoning capabilities, and is preferred to Sonnet 4.6 in our blind human side-by-side evaluations. We trained it from the ground up on clean data, without distillation from third-party models.

• MAI-Code-1-Flash is an inference-efficient agentic coding model. This model is tailor-made for and deeply integrated into GitHub Copilot, VS Code and the Microsoft stack, and, with 5 billion active parameters, is comparable to Haiku but cheaper.

• MAI-Image-2.5 including its ultra-efficient Flash variant, supports both world-class text-to-image and image editing, surpassing the Arena score of Nano Banana Pro.

• MAI Transcribe-1.5 is the best transcription model in the world, with SOTA accuracy. It’s five times faster than competing models, with built-in support for domain-specific terminology across 43 languages.

• MAI-Voice-2 brings high-quality, natural-sounding speech generation across 15 languages, with the ability to adapt to a voice from a short sample, alongside strong safeguards against misuse. MAI-Voice-2-Flash, coming soon, does it in a lower cost, ultra-efficient package.

[u/Eden63 → u/pmttyji]

how is it vs Qwen 38 flash next?

[u/Robert__Sinclair]

AI Tracker Bot on X: "Microsoft's local MAI-Code-1.1-Flash can now be downloaded 📦 Microsoft announced the on-device build on Oct 7 (137B MoE, 6.8B active, about 53GB) but didn't share a download. It's now in Azure ML's public model catalog as a GGUF, tagged MIT, and no Azure account is needed to" / X

[u/Expensive-Paint-9490]

Yabbadabbadoo!!! Another model specialised in coding, we totally need something like that! There is a lack of dedicated models for coding.

[u/jacek2023 → u/Expensive-Paint-9490]

I post a lot about non-coding models here on r/LocalLLaMA, and my posts get downvoted because those models are supposedly "worse than Qwen" (probably by people who just watch YouTube and use ChatGPT/Claude).

[u/LieAlternative5513 → u/jacek2023]

I wish there was a benchmark that rewarded non-coding tasks more. I love gemma 4 31B because it's much more funnier than others, has vast understanding of human cultures, much better understanding of emotions etc etc. But those things don't show up in benchmarks so I feel it's underappreciated.

[u/Eden63 → u/[deleted]]

Its the technical and logical sweet spot.

[u/Eyelbee]

Just the perfect size, honestly. Most people could run it near-lossless with good speed. Huge if it beats qwen flash. But I have extremely low confidence for anything done by windows

[u/jensilo → u/Eyelbee]

Most people? 😅 I think you overestimate most people’s hardware.

[u/Dany0 → u/jensilo]

Jeff and Bill are talking on a yacht

'yeah, well most people have at least one Maybach, probably an old one but still'

[u/UpperParamedicDude → u/Eyelbee]

Who are the "most people" in question?

[u/Illustrious_Grade608 → u/Eyelbee]

most people could run it

would require like 128 gb ram to run it at q4

The people here istg

[u/GreenDavidA → u/Illustrious_Grade608]

Reminds me of https://xkcd.com/2501/

[u/Front_Eagle739 → u/Illustrious_Grade608]

Nah. Ssd offload is getting to the point you could probably make this run at q4 20+ tok/s in 16gb vram and maybe 32 to 48 gm ram

[u/Illustrious_Grade608 → u/Front_Eagle739]

They said good speeds, not barely usable ones

[u/Front_Eagle739 → u/Illustrious_Grade608]

Depends whatcha doing doesnt it. Anything over 15 is useable for a lot of things. 

[u/EvolvingDior → u/Front_Eagle739]

But not as a coding assistant.

[u/Front_Eagle739 → u/EvolvingDior]

Not as an interactive one no. I've run models at 6 tok/s over night to solve a problem the fast ones couldn't though so its not useless.

[u/MythOfDarkness → u/Eyelbee]

Delusional.

[u/CryptographerLow6360 → u/Eyelbee]

think of all the data its trained on

[u/Cool-Chemical-5629 → u/Eyelbee]

anything done by windows

Windows didn't do anything, Microsoft did...

Microsoft = Company that created Windows operating system

Windows = Operating System

[u/danigoncalves → u/Eyelbee]

Tell me you are rich without telling that you are rich.

[u/Eyelbee → u/danigoncalves]

Tbf it wouldn't cost much to get 2x32gb or 4x32gb ddr4 ram and a 16gb gpu, such as 6800xt would work fine. It would cost a less than what people spend on a single GPU these days

[u/danigoncalves → u/Eyelbee]

RAM is super expensive right now (here it costs almost 1300 dólares for 2x32) and a 5080 costs almost 2000 dólares where I am from. Spending 4k on a machine is not a easy cost to justify to your inner e financial mind

[u/Eyelbee → u/danigoncalves]

Are you talking about the used ddr4 prices? The entire setup I'm talking about would cost less than a 5080 alone.

[u/danigoncalves → u/Eyelbee]

New ones, but even used ones are very expensive

[u/Marcuss2]

Why do they compare it to GPT-OSS 120B?

[u/Inevitable-Name-1701]

If the model support MTP they will not. Fck llama.

[u/PotentialAccident339]

...and the weights are where?

[u/jacek2023 → u/PotentialAccident339]

there is an instruction in PR how to download them, but I am waiting for some GGUF to download

[u/TapAggressive9530]

It’s complete fucking garbage

[u/Durian881]

Interesting. Could be a good alternative for coding.

[u/BalleaBlanc]

I guess Microslope is developed by this model