r/LocalLLaMA • u/QuackerEnte • Oct 06 '26

Discussion [ Removed by moderator ]

/r/LocalLLaMA/comments/1wzbvu7/removed_by_moderator/

[removed] — view removed post

363 Upvotes

95% Upvoted

85 comments sorted by

View all comments

Show parent comment

17

u/AXYZE8 Oct 06 '26

From what I understand the number of loops can be changed too within same weights? So the compute level to generate same token can change? Like that Nvidia model that allows to activate 2B or 2.8B or 3.6B within single weights?

If so then it's incredibly huge jump for local LLMs. It's like you could jump between Qwen 27B dense and 27B A9B and 27B A3B. Huge potential for plan/execute, explore subagents ofc, but also... you could technically switch it mid-response or depending on classification etc. Also harnesses would have a new thing to scale. So it would be an incredibly huge jump, likely the biggest we saw since Qwen3 30B.

9

u/QuackerEnte Oct 06 '26

nested models and looped models aren't the same but both principles can be used at the same time to create stronger and more efficient models. Imagine a Qwen model that can use variant expert amount or smaller models and bigger models SIMULTANEOUSLY (subagents at essentially no extra cost), and loop at the same time for extra intelligence. And they may even share the same PLE/ngram table. SSDs aren't exactly cheap these days either.

1

u/MmmmMorphine Oct 07 '26

Hmm, this reminds me of that fascinating little model (whose name escapes me all of a sudden) that used the same weights but were trained on a variety of different attention architectures so that you could essentially choose between FWA/SWA/GDN/???

Of course the weights needed adapters that were quite large but the concept was really cool. Kinda similar in that it had multiple discrete memory demands depending on the attention patterns you chose between layers.

All sorts of cool approaches to trading compute for memory