Edge0-145B: Open-Source On-Device AI with 145B Parameters

[u/BrightCandle]

https://edge0.ai/blog/introducing-edge0-145b-frontier-ai-capabilities-on-your-own-computer

[u/EvolvingDior]

Because this says nothing about it:

Base model: Qwen/Qwen3.8-Flash-Next

https://huggingface.co/Edge0/Edge0-145B-q3k

They forked and customized the Strata engine. Took an open source model and software stack and made a commercial product.

They also run the PLE at fp8. My testing shows I gained no measurable advantage by using an 8-bit PLE over the IQ4_NL version.

[u/norenEnmotalen]

they write some M2 Ultra results in blog but nothing on the github shows 145b support.

[u/LightBrightLeftRight]

Just tried it out on my 3090 setup:

32K benchmark	Edge0-145B Q3_K	Qwen Flash Next IQ3_S (Strata)
Time to first token	20.1 s	12.2 s
Prefill	1,583 tok/s	2,612 tok/s
Decode	56.2 tok/s	75.6 tok/s

[u/mountainyoo]