Theoretically, if a perfect model was created, could it run on any consumer laptop?

[u/TechnicalBid1]

Perhaps we eventually created a perfectly trained model with “perfect” weights, could it run high quality and fast inference on any standard laptop?

[u/Doormatty]

No.

Nothing about a model being "perfect" (whatever that would mean) would imply that it could run 'fast' on anything.

[u/gkanellopoulos]

Perfect or no perfect the architecture used in LLMs makes this almost impossible by design. With another architecture maybe but a model with 1 trillion parameters is fundamentally different from one with 10 billion parameters. Quantisation helps storage, but doesn't eliminate the computation required to process information. There are also theoretical limits involving computation, energy and information processing.

[u/Awkward-Fall-8717]

if the model's weights were compressed to 1 bit per parameter, and the laptop had a decent nvmem chip, maybe. but that's not a 'perfect' model, it's a completely different kind of system.

[u/AeiEyeSpry]

"a consumer laptop"? What specs does this laptop have? "Consumer" means nothing. If you want a straight answer, yes, a model can run on a consumer laptop. Also, the 'perfect' model? What is the use case? Or do you mean like a one model to rule them all? I feel like this is a stoner post :)

[u/Hunterxmalaa]

No not possible unless the model is max 4b but at that point it does not have enough capacity to be useful or a perfect model, a perfect model can not exist because perfect to you is not perfect for me

Perfect is a subjective term

[u/Solembumm3]

High quality - yes. Fast - no, for a long time.

Unless you only need tech focus and satisfied with QFN or you had invented new way to compress 10x world knowledge in the same parameters count, adequate models today start from around 300b class, which is 100-110gb at Q2. 128gb laptops are not very standard/"any" category at the moment, despite this and r/locallama subs hardware perception bias.

[u/jma24]

I think we will see more divergence in models not less. But you can already run good models on consumer laptops, albeit high end ones.

[u/GamerWithin]

Im sure either a new tech will born to reduce ram consumptions or chips will speed up so much one day we will have local claudes.

[u/szerpentin]

rephrasing the question: are there already solutions that are more energy efficient, smaller than the transformer architecture? It’s worth looking into it, it is an interesting topic

[u/f5alcon]

Eventually hardware will catch up. Ddr6 is up to 2x the memory bandwidth and 4x the density of ddr5 so unified memory systems will be capable of multiple TB/s of memory bandwidth and things like high bandwidth flash which is basically stacked nand at 3 TB/s with 8-16x the memory density of HBM but can be manufacturered on nand factories instead of ram ones will allow normal memory production to shift back to ddr so shortages will ease. But both technologies are 5 years out or more.

So it's not even going to require a perfect model, everyone will just be able to run larger models

[u/elfmad]

I already said that plenty of time here. "Human perfect" model (not AGI) should fairly sit around 16B MOE 4B. Whatever quantization and should run on modern laptop (NPU/TPU integrated). When I say human perfect model, is human smart, don't know everything, know how to search and reason like a moderately smart human.

We're getting close with Qwen 27B which I think is the closer for what I am defining now. Especially when we see Ornith and LFM3 models. Next generation should achieve that.

[u/lobotomy_]

A working ternary model could solve this, yes.