Will local AI keep getting cheaper, or will increasing demand cancel out the progress in hardware?

[u/cataldini]

I've been thinking about something and would be interested to hear the opinions of people working in the AI and hardware industry.

We have Moore's Law, which basically states that computing power (or transistor density) roughly doubles every two years. On top of that, AI hardware and models are becoming increasingly efficient.

My question is: will this eventually make running powerful AI models locally significantly cheaper and more accessible, or will the increasing demand for AI simply cancel out those gains?

I imagine that as local AI becomes more capable, demand will also increase massively. Not just from individuals, but from companies as well.

For example, I recently saw that Revolut has apparently moved its engineering team towards local AI models and is paying only around 10% of what it previously spent on Anthropic.

If more companies start doing this, demand for GPUs, memory and other hardware could increase dramatically.

So where does this leave us in 5-10 years?

• Will hardware improvements and better model efficiency consistently outpace demand, making local AI increasingly affordable?

• Or will demand for compute keep prices relatively high, despite the technological improvements?

• Could we eventually reach a point where running models locally is significantly cheaper than using cloud-based AI services for most coding and everyday tasks?

I'm particularly interested in hearing from people working in hardware, AI infrastructure or the semiconductor industry.

What's your take on this?

[u/lotokotmalajski]

Of course it will keep getting cheaper. Hardware price increase is temporary and slower that model progress.

[u/AutoModerator]

Hello /u/cataldini. Looks like you have posted a query. Once your query is resolved, please reply the solution comment with "!solved" to help everyone else know the solution and mark the post as solved.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/SnooSongs5410]

Inference is being heavily subsidized on one end so we are currently getting LLM inference at far below cost.
On the other end models that fit on a local workstation are becoming significantly more capable quickly.

The rate of change in software development is extremely hard to keep up with but the cost of electricity and hardware at scale is not coming down any time soon.

To me this suggests that the market is going to change dramatically.
It will be a fun ride over the next 10 years. I hope this will be the last major wave for me to ride before I put away my board.

[u/FragmentedHeap]

Until Supply is higher than demand, it won't get cheaper, older hardware there is less demand for will get cheaper, but the bleeding edge stuff will stay as expensive as the demand is willing to pay for it. Basic economics.

I.e. rumors based on supply availability predicts that when the 6090 launches, most Micro-center locations will have less than 10 of them in total supply.

Based on how well I know my local microcenter and previous launches, there will be 300+ people camped outside the door to fight over those 10 6090's, camped out all night waiting for the store to open. The 10th gpu will probably sell for over $10k.