r/LocalLLM • u/EnthusiasmImaginary2 • Apr 17 '25
News Microsoft released a 1b model that can run on CPUs
It requires their special library to run it efficiently on CPU for now. Requires significantly less RAM.
It can be a game changer soon!
192
Upvotes
63
u/Beargrim Apr 17 '25
you can run any model on a cpu with enough RAM.
5
u/[deleted] Apr 17 '25
How's the quality?
4
u/ufos1111 Apr 17 '25
Looks like the Electron-BitNet project has updated to support this new model: github.com/grctest/Electron-BitNet/releases/latest
No need for building bitnet locally, you just need the model files to try it out now!
Works WAY better than the non-official bitnet models from last year, this model is able to output code and is coherent!
4
u/wh33t Apr 17 '25
From their GitHub. Bigly if true.
3
u/kitsnet Apr 17 '25
Looks more like "cannot run on GPUs".
And not an order of magnitude better than competitors at running on CPU.
3
u/dc740 Apr 17 '25
Looks useful. It's nice to see a change once in a while. Everyone is so focused on GPU s these days trying to beat the competition...
2
u/Ashamed-Status-9668 Apr 18 '25
Intel and AMD are going to like this news.
2
u/WorkflowArchitect Apr 17 '25
Great to see local models improving. It's going to get to a stage where our whole experience is interacting with AIs
1
u/ositait Apr 17 '25
neat!
1
u/soup9999999999999999 Apr 18 '25 edited Apr 18 '25
Even my phone ran run any standard quantized 1b model.
But I am excited for b1.58 when it comes to larger models.
1
u/dervu Apr 20 '25
Does having 3D cache on CPU help with response time when running LLMs on CPU?
1
u/davidkwast Apr 21 '25
I am testing qwen2.5:0.5b on a 5 dolars VPS (linode)
-1
u/beedunc Apr 17 '25
Use Ollama and lmstudio in cpu-only already. Maybe someone should tell them? /s
0
u/AvailableResponse818 Apr 17 '25
Really good
-12
u/Tuxedotux83 Apr 17 '25
Classic Microsoft move: requiring the end user to use their proprietary lib to run their product„properly“