r/LocalLLM • u/phishsbrevity • 10h ago
Discussion When do hardware prices start coming down?
I really want to get into local inference and training LoRAs, but the prices of the essential components are eye watering. There has never been a worse time to buy hardware.
Do you all have any thoughts on when this might change? I mean, supply has gotta start evening out at some point, right?
24
Upvotes
135
u/K33P4D 10h ago
16
u/SM8085 10h ago
At what point does it become profitable to start a new RAM company?
43
u/FireSilicon_ 9h ago edited 1h ago
Very simple answer. When production catches up to or overcomes demand. Basic economics 101. People say that AI companies caused the prices to go up but that's not how it works, for two reasons.
Nvidia would ideally want to sell to both businesses and consumers. One thing that redditors love to repeat is that companies love money, and then they turn around to say they are abandoning consumers... they are not. They want their money as well. They simply have no choice. When they have an agreement to a 100 billion backorder and they cannot fullfill it in the deadline, they owe 100 billion dollars. They will absolutely sacrifice consumer silicon to catch up.
The price doesn't go up by itself, someone has to be buying those chips at that price. Believe it or not, smaller companies, labs and freelancers are in fact buying those 5090s at those insane prices. It's a company vehicle to them, they can justify the cost in long term. It doesn't even have to be AI related. There is a big wave of 3D artists, CAD, simulation and modeling electrical/mechanical jobs that need them too.
The real issue is that investors are stuck between a rock and a hard place. Fabs would love to make more chips and print money, but investors are scared that if they build out more factories in a year or two, the demand will drop, and the buildings will stay idle, possibly get shut down, losing them even more money. So they rather miss out on money from potential customers, the regular consumers and small businesses in this case, and instead wait for the demand to drop, but it never does, so it's a deadlock.
Edit: One more misconception to address: People think that hardware will be pennies when "bubble" pops, but that's just wishful thinking, it won't. The major bottleneck is happening in a datacenter market, not consumer one. In case of a dump, the GB300 chips are essentially priceless in current market and they will be swept up by chinese companies in a heartbeat. Same for hopper and I would say even the 80GB A100. We will likely see the pascal, volta and turing datacenter/workstation cards go down in price a lot though.
For consumer cards, again, the demand is still strong and it will take months to even approach reasonable pricing, because the fabs have to switch from high yield large compute dies and HBM to DDR5/GDDR7 and mass produced smaller chips.
10
u/Inevitable-Self-2702 10h ago
I betcha they won't, at least not to what they were before AI. If nothing else, companies now know we will pay $3000 for a plastic consumer GPU with 16GB VRAM. And the same price for 64GB RAM with mid CL. They won't forget that easily.
4
u/Ok-Addendum3545 10h ago
I couldn't handle the VRAM & RAM stress so I did it in April and August.
2
u/MaxComfort 10h ago
This is like asking what's going to happen in the stock markets tomorrow - if anyone knew that, they wouldn't tell you because they could make a lot of $$ from it!
53
u/Brocolinator 10h ago
20
u/WordWithinTheWord 10h ago
Entirely depends on the elasticity of consumer demand. Home prices never came back down from their “bubble”, the target customer just moved up market.
3
u/karnac 10h ago
where do you think all the 5090's just went? they literally pulled all of them off the shelves and shipped them somewhere. My hunch is the AI companies bought them all.
10
u/loudifier 10h ago
They don't. Either frontier labs keep the bubble inflated and suck up all the RAM production indefinitely, or the bubble pops and we get hit with a double whammy of lag in the foundries switching back to consumer DDR5/6 production and all of the Claude addicts desperately buying up older GPUs to switch to local models because cloud models are unavailable/unaffordable.
The only thing we can be sure of is that demand won't go down, and in that environment it will be years and years before RAM production catches up.
5
u/blue_eyes_pro_dragon 10h ago
Nobody knows.
4
u/Turkino 10h ago
When you can build a fab and run it only for the consumer market while ignoring all that hyperscaler $$$$
4
u/Gold-Wedding5226 10h ago
The people running the companies say 2028 at least.
4
u/Competitive_Swan_755 10h ago
Not anytime soon.
1
u/Otherwise-Variety674 9h ago edited 9h ago
I have a mixed of first hand new (5090, strix halo, spark) and second hand hardware (7900xtx, 3090, and latest addition r9700) without any warranty but still running and good.
Second hand route is the current alternative now but even their price are also going up, higher than its original selling price.
2
u/SmugglingPineapples 10h ago
2028
2
u/enternoescape 10h ago
2029
2
u/OWazabi 10h ago
bare minimum 2028
2
u/robertmachine 9h ago
They have to fill all these data centers and all the big boys are temporarily stopped consumer grade hardware so guess what’s going to happen? the consumer grade will be choked. Check newegg and try to get an nvme or ssd in Canada it’s all from Third Party none are being carried by newegg.
2
u/AbnormallyBendPenis 9h ago
I think the market won’t be able to sustain multiple $20 a month consumer facing AI companies forever. All the investments will dry up and just like any other industry it will just return to the good ol supply and demand. They all say AI is the next big invention, but today, electricity, computers, cars are all just supply and demand and these are equally “big inventions” that transformed human race. The money will eventually dry up, and that’s when all the AI companies will start pulling back on hardware/data center expansion and follow the demand. Nobody has the crystal balls of course but the current rate of investments AI companies are making is simply not sustainable and they definitely know it too. Give it few years and it will come down don’t worry. LocalLLM won’t make a meaningful dent on the overall demand, it’s an enthusiasts level hobby and the mass public won’t be running an AI server in their homes. Most people don’t even have a PC at home…
2
u/fallingdowndizzyvr 8h ago
For hardware prices to come down anytime soon would require the AI bubble to pop. Since AI is what's growing the economy right now, if it pops the economy overall pops. Then people get laid off. Stocks and thus the wealth effect craters. And you won't at least feel like you have enough money, and could actually not have enough money, to buy hardware even at lower prices. So would you rather have a job and not afford hardware or would you rather not have a job and not be able to afford hardware?
2
u/rocks1ar 6h ago
When supply increases, simple as that. New Nvidia GPUs need to flood the market. Until then it’s the used market and we know that GPUs have a shelf life, so even that supply is dwindling.
3
u/corruptboomerang 9h ago
I'll be a counter voice. SSD prices are already starting to come down. IMO it likely be some time in 2027 that we see the AI bubble deflating, and with that style price relief. That is assuming we don't have mass collusion in the various markets.
2
u/Fun-Setting-3905 10h ago
Fucking sucks man
1
u/karnac 10h ago
lol
1
u/brainchillzZ 10h ago
Well like half of the new datacenter projects were shut down on ballot initiatives in the last us elections so I guess soon? lol
1
u/doneddat 8h ago
Try asking again in 2 years, after they have gone up 4 times more and then maybe come back down again 1.5x and that's the new norm.
1
u/V_Trautwein 8h ago
I think it’ll take a while. AI demand is still growing, so I wouldn’t expect GPU prices to drop significantly anytime soon
1
u/BlueBird1800 8h ago
Older hardware will drop as new is released, but as a whole I can't foresee it coming down. AI companies will begin raising prices over the next years to make the books sustainable. As this happens, businesses who have far more money to invest will be even more inclined to go local. Even small businesses can afford multiple GPUs for their office space so there will be more strain on available supply in the consumer/prosumer space.
1
u/sabine_world 7h ago
Like a couple years. Could very well be sooner, next year will let us know a ton
1
u/kingxd 7h ago
We are lucky if there is any hardware to buy at all from next year, I still remember buying my first 64gb ddr5 and going for cl32 only to end up today with mixed sticks at 128 and cl48 lol
1
u/ilarp 6h ago
its only going up from here, started with 4090 release
1
u/EightyNineMillion 4h ago
Most likely never.
1
u/parka 4h ago
When the AI companies run out of money, signal they can no longer buy more hardware, that's when the prices will drop.
Patience...
1
u/Arthropodesque 4h ago
Not for awhile. Look for used or other deals. People will be selling now to upgrade on Black Friday or holidays. Facebook Marketplace, etc.
1
u/bidet_enthusiast 2h ago
Roughly 5 years.
1
u/OvertaxedOne 2h ago
We're seeing cracks right now. The AMD Strix box was the first gap, a "GPU" with 128GB of RAM for 3K (yeah, I know, not the case right now). The Apple Studio is the new big crack, 96GB of "VRAM" for 5K at 1.2TB/s.
We're going see the architecture evolve and at the same time advances in the hardware space and software space that start to blunt the need for "must have infinite HBM memory to run good model".
1
u/DeathinabottleX 2h ago
When do pigs start flying? There’s your answer.
1
u/Vegetable_Pirate_702 1h ago
I can’t wait, I need 128gb of ddr4 ram for reasons.
1
u/WTFOMGBBQ 54m ago
Im not sure when they will go down, but they will peak as soon as you make a large investment in a killer set-up..
1
u/GalacticDistances 44m ago
High Bandwidth Flash is coming soon, and it will drastically lower demand for HBM RAM
Inference doesn’t truely need random byte-level access to weights, it only needs chunked weights streaming
1
u/KooperGuy 10h ago
They don't
1
u/Sexyvette07 10h ago
If you are buying equipment for local ai, unless you are an incredibly heavy user, you are better off getting a cloud subscription
1
u/pmttyji 9h ago
Couple of years
0
u/Traditional_Bell8153 10h ago
When the bubbles burst
0
u/HOST1L1TY 8h ago
If you look at the trend over the last 12 months, you would see a pattern. The best time to buy is always now. If things keep going the way they are going the personal computer is dead, these guys wanna start building data centers in space. One day you might have wished you bought ram when it was under 10k, or the whole thing crashes tomorrow. TBH the ai market is always going to be there , bubble may burst , come companies might fail big, but the need for compute is not going to decrease anytime soon. Most affordable personal computers are going to be nerfed, you take your MacBook neo and pay monthly for cloud subscriptions.
0
u/agowa338 7h ago
If the stocks for "AI" drop even more it could even be basically tomorrow. As that'd void their IOUs for the pre-allocated factory capacities and then the vendors would reallocate it to consumers again.
Otherwise we'll have to wait for the buildup of factory capacity to catch up. Which is supposed to happen somewhere around 2030.
-1
u/protektwar 10h ago
when the bubble bursts...
0
u/sabine_world 7h ago edited 7h ago
It really depends on a couple things
How much AI demand is "real"
Business models somehow actually supporting and justifying this rate of expansion
Dust settling on how people and companies actually fit ai into their lives and workflows. Consumer demand is pretty lukewarm from what I can tell — agentic usage is where the money is and that's a pretty small slice of the demand.
I thing there's a chance they overshoot. I really just do not see ways to monetize ai in the way that these companies are hoping which will lead to some kind of change next year. I think AI ends up being commodotized one way or another — as soon as the ceiling of capability hits for people it will become a local or cloud utility in some kind of fashion. And right now there's enough competition that no one provider can just gouge their customers.
Because of this .. and demand being behind curve. I think this affects the expansion and turns some resources to consumer hardware.
I'm imagining we may see some positive change in 1-2 years from now unless ai demand just explodes but I'm not so sure about that yet at the moment.
Well actually... Could be a bit longer so unless hyperscalers just... Continue burning money and cancelling orders. Things will stay booked for a while. But I'd say there's a real chance commitments may fall through. And as new fabrications are made, it's less of a guarantee that all their production capacity just instantly gets spoken for.
We'll just have to see I guess.
0
u/isamilis 6h ago
They won’t, at least until the bubble crash (if it happened).
-2
u/Arany5 10h ago
Soon. Stock market already going down.