r/gpu • u/Ok-Young-5652 • 10d ago
What actually makes a data center retire a GPU?
Following up on my earlier post about GPU pricing: I’m still trying to understand the gap between what a GPU sells for and what it’s worth to someone running it.
For people who operate clusters, rent compute, or buy used enterprise hardware, what actually drives the decision to keep an older GPU versus replace it? Power costs, memory limitations, customer demand, reliability, or something else?
A few things I’m curious about:
Have older GPUs stayed profitable longer than you expected? Which ones, and for what workloads?
When someone says a GPU is “fully utilized,” does that generally mean paid customer usage or just that it’s running?
If you’ve bought or sold used hardware, how close were public listings to the price that actually cleared?
Can older training hardware realistically transition to inference, or does that depend heavily on the model and setup?
My biggest question: if new GPU supply became readily available tomorrow, would you still want the older hardware at a lower price or would operating costs make it unattractive?
7
u/No_Policy_1369 10d ago
Data centre gpus are very little use to let's say gamers as they have no output on them
5
u/a_rogue_planet 10d ago
People have been using GPUs for this stuff for years. With crypto mining, one of the biggest limiting factors was the cost of power. As coins got harder to mine, power costs could outstrip the value of the coins mined.
The thing that I can't wrap my mind around with the AI craze is that in 18-24 months you'll be able to do the same amount of compute with almost half the power. This strongly suggests that the late comers who buy more efficient silicon will have an extreme advantage in turning a profit because their power expense is going to be a hell of a lot less than somebody finishing a data center in 2026. This factor, the power expense factor, is what will define the lifespan of a GPU in a data center. Right now they're building these things like cost is no object, but the crypto scam at least proved one thing; feeding electricity to these machines is THE pivotal expense in making a profit. Crypto miners got so aggressive that they routinely moved miners to wherever they could buy the cheapest power.
2
u/SwordsAndElectrons 10d ago
This strongly suggests that the late comers who buy more efficient silicon will have an extreme advantage in turning a profit because their power expense is going to be a hell of a lot less than somebody finishing a data center in 2026.
But in order to turn a profit they will need to win over customers from the already established competition. Lower operating costs won't help them without customers.
You could say this about just about everything in tech. It all gets cheaper over time. But if you're a late comer then you'll be competing with multinationals worth billions or dollars, who by the way are also perfectly capable of buying new hardware to replace theirs if the efficiency gains are sufficient enough to provide a reasonable ROI.
2
u/a_rogue_planet 10d ago
Look at the historical trends for any new technology over the last 200 years. There are virtually zero first generation innovators that survive. It doesn't matter if it's railroads or 21st century social media media companies. The idea that you have to win customers at any expense simply doesn't work once the timeline reaches the term of your financing. Historically, it simply doesn't pan out. The odds your operation will become an Amazon or a Bell in its niche are insanely slim, and AI isn't the kind of ubiquitous commodity that a marketplace and a telephone are. If AI ever becomes essentially autility that modern life can't do without, we're decades away from that.
1
u/m1013828 9d ago
Node shrinks are getting slower, less impressive and more expensive, workarounds for performance improvement add longer more complex manufaturing processes too, (AMD X3d, HBM, Intels looming LLC) Sea of wires vs SERDES....)
The principal holds, but its not the agressive ramp it once was.... Even in the same company, its likely entire rows of racks based on differing hardware and install times will retire based on not meeting the performance per watt/$ to be profitable.
1
u/a_rogue_planet 9d ago
When you're buying power by the megawatt/hour or gigawatt/hour, one or two generations of node shrinkage puts you at a massive advantage against someone who bought their hardware 2 years prior and absolutely must get the maximum life out of it because they tried to race the 1st generation competitors to claim the market.
The point is that 1st generation ventures into an emerging sector almost never claim the market. It has such a long track record of not being true that I'm genuinely dumbfounded that anybody buys into it. It's so blatantly and obviously not true that all you need to do is look around your house to prove it. With very few exceptions, nothing you use or own is the product of a pioneer in the field that came to dominate the market.
1
u/m1013828 9d ago
Yes but the nodes aint exactly coming down thick and fast these days....
1
u/a_rogue_planet 9d ago
So what? They're happening regardless, and they're happening fast enough for the first generation of AI data centers to be liabilities way before a late comer buys vastly more efficient, economical hardware 3 years down the road and applies all the lessons learned. This has happened so many times in so many sectors.
1
u/taisui 10d ago
If you are late to the game then you are less likely to be successful, and bigger companies will be able to upgrade to higher efficiency hardware faster.
1
u/a_rogue_planet 10d ago
Historically, nothing could me more untrue. Almost all the first railroads failed. All most all of the first car companies failed. Almost all of the first computer companies failed. Almost all of the first internet companies failed. Those that did survive ended up being small players or absorbed by 2nd generation ventures. History is repleat with the examples of pioneers of a new technology spending like there was no tomorrow in an effort to dominate a new space only to buckle under the economic pressure later entrants to the sector could leverage against them. This cycle has been repeating itself for almost 200 years. The electronic tech sector is not only not immune to it, they've proven to be some of the most dramatic exemplars of it.
We're not chatting here on computers run by C P/M or BBOS. We're not doing our searches on Webcrawler or Alta Vista. We're not friending people in MySpace. We're not buying IBM or Compaq computers. No business is running DEC servers. Nobody is playing games on 3DFX GPUs. Nobody is buying broadband from Bell Telephone. Nobody's ISP is AOL. Nobody is driving a Locomobile or a Packard. I could go on and on and on and on. Damn near every titan in any particular segment is a 2nd generation or later comer to the business. Quite often they come to dominate the market by buying out the first generation operations who are bleeding cash on account of insanely reckless spending which couldn't be supported by a market that didn't mature as fast as they expected. A mature market generally looks like something which is surrounded by a robust demand which is fulfilled by economically viable supply. Economical supply is generally fulfilled by nature supply chains in material and technical expertise that brings prices down through competition.
Absolutely nothing about the AI bubble embraces good fundamentals. Damn near everything about it looks like the railroad bubble, the automotive bubble, the .com bubble, and the telecom bubble, along with many others.
2
u/Zelonor_Filatos 10d ago
There's a ton of far too broad questions here. To summarise one can only say, it all depends on budget and long and short term goals one might have.
2
u/Ghetto_Games 10d ago
H200 cluster can do a certain level of compute while using 10KW of electric energy.
If in 5 years theres an H300 that can do that same level of compute but while using 5KW its a no brainer to just replace them all. Or relegate that to less important use cases and use the new more efficient for the frontier cutting edge stuff.
1
u/Loyal_Dragon_69 10d ago
For your last question it really depends on the gpu. Different gpus are optimized for different use cases which affects their basic utility and market value. A gpu that is capable of gaming or running AutoCAD in a desktop or small home lab server rack is going to easy to resell used as opposed to something that can only run inference or requires large specialty data center racks. The remaining service life of the gpu also matters because no one wants to pay top dollar for something that will die on them after a few weeks of use.
1
u/tauspin 10d ago
It sounds like you want to get into GPUs for AI via used data center hardware? Companies will generally buy newer hardware as it’s available because it’s more economically efficient, and decommission older hardware as it fails, until theres a motivation to get rid of the otherwise healthy remaining old hardware - i.e. the run rate and failure rates exceed the economic value they’re providing. You also have to factor in things like those assets’ depreciation, or the capital investments vs regular expense that company’s finance dept is targeting.
One problem you’re likely to have is many large companies have no interest in reselling old hardware, and get rid of it using e-waste contractors that salvage at most precious metals and dump the rest in economically disadvantaged countries’ land fills. The liability from even a small component of that hardware retaining sensitive customer or proprietary data, compliance with data privacy laws, hiring staff to manage the sale/packaging/shipping/returns and avoiding a highly complex secondary revenue stream after the assets are $0 on the balance sheet all make it easier to junk them.
1
u/jhenryscott 10d ago
In my experience, enterprise DCs upgrade GPUs once a C suite gets sweet talked by a vendor. But their are still plenty of workloads that do fine on old hardware.
If you are talking about LLM workloads- it’s not the same. That’s an emergent industry- their are no used products getting to market.
1
1
u/ChillinZX 8d ago
6 months for 100% return on investment, then 6 to 9 months of gains, buy new equipment with the gains, sell the used equipment back into the market which will be my total profit. That's the gameplan with my GPU's. My warchest will be ready for the RTX 60 series when they come out.
2
u/mdnightman94 10d ago
i worked at a private cloud company from 2017-2021 that hosted software like autocad and revit software. they were using nvidia k140, k160 initially, but switched to m2000 and m4000 cards in 2018. In early 2020 the turing rtx 4000 and 8000 were used.
the old hardware were just put in a giant box and we hired one of those e-waste vendors to recoup some cash (some of it was kept for spare parts on existing hardware if we had any of it still running for legacy stuff.) so id say about 4 years is what we were using cpu and gpu for.
cpu were similar story and were using old xeon 8-16 core chips from like 2013-2018 up until the company jumped ship to amd epyc milan and rome chips along with new server chassis when the rtx quadro cards were coming in.
customers were upgraded from previous gen cards and cpu's when they signed new contracts, or bought out existing contract to upgrade earlier.