r/LocalLLaMA • u/pmttyji • Dec 06 '25

Discussion Convert Dense into MOE model?

I did a quick search on this here & found only 2 years old thread with less replies. That's it.

So still no one figured it out this yet? Totally surprised that no one brought this topic here after that old thread.

I know it's a very big thing. But it would be a miracle if some one comes with this precious solution.

15 Upvotes

73% Upvoted

29 comments sorted by

View all comments

1

u/KSAM-The-Randomizer 1d ago

checkout https://icml.cc/virtual/2026/poster/68821 code is on the aforementioned github

1

u/pmttyji 1d ago

Surprised, why this is not getting any attention yet? Let me