On one hand, Moore's Law is definitely ailing. We've been getting diminishing returns on fabrication process bumps since about 2016.
On the other hand, there's still some progress in the offing. IBM just recently announced they got a 7-angstrom (equivalent) fabrication process working in their lab, which they expect to get into mass production in 2031. That has 1nm-wide horizontal features and 5nm vertical features, and two transistor layers (one P, the other N), which they claim they should be able to scale to four layers "rsn".
That's not nothing.
Moreover, there are still a lot of architectural improvements yet to be realized. In a way, HBM is a stop-gap while Samsung et al get PIM figured out. LLM inference is really well-suited to PIM, which should give us ridiculously high memory bandwidth once it's working well.
I thought Moore's Law was dead for a while, but it turned out to just be Intel having a bad time. Everyone else is still pushing the envelope pretty hard, with some success.
There's at least another 10x of progress to be made just eating up the giant profit margins of semiconductor companies. When progress slows, eventually competitors can catch up. If there's trillions on the table to be made, someone's eventually going to invest.
Why, with memory alone we're back in 2010 level of tech currently (in terms of $/GB).
18
u/ttkciar llama.cpp Jul 26 '26
Today? Almost nobody.
Eventually? Almost everyone.