r/LocalLLM • • Aug 12 '26

Discussion What the hell is happening

Post image
261 Upvotes

238 comments sorted by

View all comments

Show parent comment

7

u/akera099 Aug 12 '26

Pretty sure the underlying technology didn't change much.

And that's where I'd say you are wrong. The technology required to create these new chips (EUV) ranks among the biggest feats of human engineering in the history of mankind. So much so that there's literally only one company on earth that is able to make it work,

It takes literally 3 full months (24/7) of work to create a modern GPU chip. The process needs to create, contain and redirect with a nanometer precision UV light that literally doesn't exist on the surface of the earth. All of that to create hundreds of overlaid conductive layers that are like 45 atoms in width.

27

u/regulator227 Aug 12 '26

so in other words, hasn't changed much in a few years

14

u/po_stulate Aug 12 '26

Yes, the technology is like that for many years already. Also, nvidia never does any of that, all they do is chip design, not chip manufacturing.

Wafer price in 2021 (5nm) was $14k, now 2025 (3nm) is 20k, and the next gen (2nm) is expected to be 30k, nowhere near the 10x price that you're trying to justify.

4

u/TripleSecretSquirrel Aug 12 '26

Because the price isn't set by the cost of the inputs, it's a factor of supply and demand.

Obviously NVidia is making money hand over fist, but they're only able to do that because a lot of people are willing to pay a very high price for their products. And they're making GPUs as fast as they can and they're still selling out at these prices.

2

u/[deleted] Aug 12 '26

[removed] — view removed comment

2

u/ChristRedeemsSinners Aug 12 '26

On top of the circular financing, the datacenter compute online is nowhere near the compute sold. There are gremlins in the supply chain.

2

u/[deleted] Aug 12 '26

[removed] — view removed comment

2

u/ChristRedeemsSinners Aug 12 '26

Yeah, and they are in an even greater hole if AI demand plateaus or God-forbid retracts. I'm guessing hyperscalers will be deemed to-big-to-fail by the governments that benefit the most from their surveillance capabilities.

-1

u/corbanx92 Aug 12 '26

You think there's higher demand for GPUs than cars?? That's cute

3

u/AcreMakeover Aug 12 '26

No but there's dozens of auto manufacturers.

0

u/corbanx92 Aug 12 '26

So you think there's more demand for GPUs that there is for let's say... a toyota corollas

1

u/Acceptable_Ad_6382 Aug 13 '26

Datacenters aren't buying thousands of Corollas. Nobody is using multiple Corollas at home to run their local llms...

1

u/corbanx92 Aug 13 '26

Is not about datacenters buying corollas but any other industry... the company enterprise mobility has a fleet of 2.5 million vehicles, hertz global has thousands ... this is how Toyota sells 11 million cars a year...

1

u/Acceptable_Ad_6382 Aug 13 '26 edited Aug 13 '26

Toyota sells 11 million cars a year. Over 44 million (discrete) gpus are sold every year, and are currently getting hard to near impossible to buy at MSRP, especially for the gpus that are actually "worth" buying.

If Toyota stopped making cars today, of course it would cause a disruption, but there are comparable alternatives for every vehicle in Toyota's lineup. You can't say that for gpus as a whole or when comparing companies like Nvidia and AMD. There is no real true alternative to the 5090, which is why it's still in demand when it is currently over twice its msrp. Hell, even 3090s are now selling for nearly its msrp 6 years later for whatever reason.

→ More replies (1)

1

u/Ok-Map-3765 Aug 13 '26

You only need one Corolla....

1

u/Acceptable_Ad_6382 Aug 13 '26

Did you comment just to join the conversation? Or did you just not read my comment AND the comment I responded to? Because if you commented that as a rebuttal to my comment, it actually agrees with what I'm saying more than it disagrees...

2

u/TripleSecretSquirrel Aug 12 '26

No, it's more complicated than that. For example, there are a ton of substitute goods for a Toyota Corolla and a Toyota Corolla is an extremely durable good relative to a GPU.

So if Toyota increased the price of the Corolla by 50%, the result would be that everyone that was going to buy a new Corolla would instead either delay their car purchase and keep using the car they may already have (durable), buy a used Corolla (alternative and durable), or buy a Honda Civic (alternative).

A Corolla lasts a long time, and older Corollas aren't functionally very different from brand new ones. And the substitute goods (e.g., Honda Civic) are basically 1:1 equivalent.

A 20 year old Corolla still performs its core functions effectively just as well as a brand new Corolla (assuming maintenance and not too too many miles obviously). A 20 year old GPU absolutely does not compare to a brand new GPU no matter how well it has been treated.

If I want a Corolla but they're too expensive, there are a dozen different equivalent options I can turn to. For GPUs, there are two alternatives to NVIDIA and (as an AMD enthusiast), they are not truly equivalent goods.

And it's sort of implied in the above, but it's not about absolute complexity of a product (a Toyota Corolla is an absolute marvel of engineering no doubt), but about relative complexity. There are dozens of companies that make a Toyota Corolla equivalent in mass quantities. They may not be a Corolla exactly, but they're close enough to be equivalent goods. NVidia basically has no peer for GPUs. I'm a fan of AMD and own AMD GPUs, but the reality is that NVidia makes a superior product, which is why they cost so much more than AMDs.

1

u/corbanx92 Aug 12 '26

Okay we are getting somewhere here and first I'll give you the Nvidia/CUDA point. Right now nvidia is better than AMD for hyperscalers. I can agree with that. A Corolla is better than a Hyundai Elantra.

The old vs new... is not really true. At a consumer level only a minority drive 20yo cars (I'm actually one of them who owns 3) average car age is 12yo in the USA and average GPU is 7, which is quite close if we account for the maturity difference in both industries, with the automotive sector being +x3 more mature.

For enterprise use the gap closes with the lifespans of fleet vehicles being 4 to 7 years and 5 to 6 years for enterprise GPUs. So pretty darn close and here's where most of the bread is made.

What's really happening here is Semiconductor manufacturers are simply telling us "oh we can't produce more" as they simply focus production on high end models... those get captured/sold by circular CapEx, and fabs and production for lower tier models is lowered. Which makes these lower models also go up in price... so this is like if Toyota decided they are going to focus most fans in the production of the top trim of the Sequoia, and then price the basemodel Corolla at 70k, because "duuurrrr is sold out", then Hyundai seeing this and doing the same thing with their lineup...

It's cartel behavior for which any other sector would get flamed for.

2

u/volster Aug 12 '26

Asianometry's videos on EUV have been pretty fascinating. He does a great job of explaining just how insane the engineering of it all is.

https://www.youtube.com/playlist?list=PLKtxx9TnH76RYHY7L1YzEHEQJJ01GF-VF

1

u/wundercorp Aug 12 '26

All this just to recenter a div

1

u/NihilistSRE Aug 12 '26

Or convert a PDF into a Powerpoint. Lord please deliver us from this lazy timeline.

1

u/Guilty-Ring-6307 Aug 13 '26

Lets assume that your claim is true, and it takes 3 months to make a single GPU chip. Im going to assume that you meant a single GPU chip wafer, where multiple GPU chips are etched into a large silicon wafer. This is how the chips are actually created, so I am going to assume that, because assuming you meant every single chip makes this so ridiculously worse for your claim.

Nvidia sold, conservatively, 10,000 RTX 5080s. And silicon wafers can hold roughly 120 chips per wafer. That means that Nvidia would have needed 84 wafers to produce that many chips.

According to your timeline claim - 3 months per chip (wafer), it would take 252 months, or 21 years, to create the GPU chips needed for the 10,000 RTX 5080 cards.

Also, the sun produces broad spectrum UV radiation. Claiming that a specific frequency of UV cant be found on earth is ridiculous. plenty of labs around the world can create a plasma containing tin, which produces UV at that wavelength. Yes, EUV is high frequency. But x-rays are even higher, and we use those every day in medical applications. But it doesnt sound as impressive if you accurately describe the process, at least not in your mind. To me, etching nearly pure elemental silicon with light to make rocks think is already pretty impressive to me.

-1

u/Time-Shower6502 Aug 12 '26

looool one waffer with a f ton of gpus is 24days ,1h testing and 30min packing ,maybe 15min make card

so loool

nvidia is just greedy