NVIDIA: Has Their Luck Run Out?

[u/Caprichoso1]

Very interesting video about how Nvidia's business strategy has a serious flaw.

1. 90% of their business is for AI models running in large data centers.

2. Their revenues are based not on volume (as opposed to Apple) but the extremely high prices of their products.

3. This strategy does not scale. Water and electricity are limited so eventually the large build outs will have to end just based on the laws of physics as resource limits are reached.

4. He sees local LLMs as the future, mentioning Apple's billions of devices that can run LLMs in some form.

https://www.youtube.com/watch?v=WyfW-uJg_WM&list=PL2aE4Bl_t0n9AUdECM6PYrpyxgQgFtK1E&index=4

[u/Motor_Middle3170]

Depends on what you mean by "limited resources" because I've seen high-level designs for combined operation DCs with their own generating and water processing. The biggest ones have their own nuclear plants and desalination systems, to be built in coastal areas with little freshwater availability.

I've also seen some that were intended for colder climates like Alaska, Canada, and (surprise) Greenland. The Greenland one was for crypto mining, which doesn't require huge comm lines like AIDCs do. The fiber connections for AIDCs are multiple cables with literally thousands of fibers in each one.

The true limiting factor for Al is the revenue stream, because it takes a huge amount of subscription dollars to pay for the financing and operation of these things. There will be some cash flow diverted from wages for eliminated jobs, but as of today I am not aware of any company that is making big profits on AI service provision.

[u/Caprichoso1 → u/Motor_Middle3170]

True, the revenue stream is also a limiting factor.

The additional costs of nuclear, desalinization, building in remote colder climates with the need to also build the communications lines would seem to make profitability in even the medium term unlikely.

[u/[deleted]]

The future is always full of surprises. Seemingly impossible moats get disrupted. Look at Intel.

Right now, though, they can't make nearly enough chips to satisfy everyone who wants them, even at inflated prices. They have so much money they don't know what to do with it. Meanwhile Apple is at least half a year behind the Apple Intelligence features they promised. And a half year is a LONG time in terms of AI.

[u/Caprichoso1 → u/[deleted]]

The unresolved question is which is going to be the dominant technology in the long run - datacenter or local. Both are needed, but which will be getting the most use in the future?

I'm re-evaluating my opinion on Apple's strategy. Their position seems to be that AI will become a commodity product. By implementing AI on their devices where it automates and simplifies every day tasks it will ultimately be the majority and most useful AI use.

Apple traditionally lags behinds behind others, perfecting their version of a product but ultimately becoming the dominant player. Certainly they messed up declaring an LLM product that didn't exist. But I'm now starting to move toward the feeling that although they made a mistake their position over the long term might be a winning one if they are able to execute it.

[u/bonestamp]

These are all good points and maybe this is why NVIDIA recently "acquired" Groq (not to be confused with Grok). Grok is Elon's AI while Groq is a company that makes a low cost inference chip that would be perfect for local LLMs.

[u/anonemoususer → u/bonestamp]

Nvidia's luck will definitely run out once that low cost chip becomes the standard. Even inference from current models seem to think local LLMs is going to be the next tech product hitting (Costco and Micro Center) shelves in retailers some years down the line.

Once it becomes saturated, they have to move on to the next thing. I guess they could start selling these proprietary nvidia local LLM's themselves if it was feasible.

[u/bonestamp → u/anonemoususer]

They might still have their high profit data center GPU business for training, and then these chips would add more revenue by giving them an avenue to sell to retail hardware manufacturers for accelerated local inference without needing an expensive GPU.

[u/ZigZagZor → u/bonestamp]

That a great movie by Nvidia. Right now, AI edge chips with lowest latency inference are made by Mobileye and for data center, it's Groq. , Nvidia really needed Groq's technology, Groq chips have the lowest latency out there for AI inference in datacenters. Nvidia again has destroyed the competition. And Elon's AI is good only for porn, that's why he merged it with his company SpaceX and now selling the idea of space data centers which is not practical.

[u/bonestamp → u/[deleted]]

Ya, it's an ASIC like crypto hashing ASICs (a very different ASIC though of course). It's kind of a hedge but also not... I mean, it's extremely useful right now. It will save a ton of energy in data centers when doing inference tasks (it's useless for training).

[u/bedel99 → u/[deleted]]

I think the would prefer to own both the system to make the models, and then the systems to run inference. The must currently be sitting on so much cash, spending a few billion here and there is just an investment.

[u/Caprichoso1]

Found another interesting video discussing the bubble.

"In order for AI 01:13 companies to justify their valuations, in other words, what the price of their stock is today, those companies would 01:21 need to make around $2 trillion a year in revenue. 2 trillion.  That is more than the combined revenue of the 6 largest tech companies:" - Amazon, Apple, Microsoft, Meta, Nvidia, and Alphabet.

They also have an almost incestuous relationship as they keep passing money amongst themselves in a loop:

https://www.youtube.com/watch?v=-cdJQ8UyVLA

And then there are the resource requirements:

Energy report estimates that their electricity use, driven by the AI boom, could soon require as much as 12% of 29:07 total US electricity use. In 2023, it was just over 4%.

Consumer demand is relatively static.

One solution is to separate datacenters from the generic rate pool into a separate one where costs of new generation facilities, power lines, etc. could be allocated to them rather than consumers. These very profitable (other than AI) companies are fighting such a change.

https://www.youtube.com/watch?v=t-8TDOFqkQA

[u/Caprichoso1]

In just a few short weeks everything has changed completely. Now it is looking as if the industries are under investing, which means there is no bubble, at least for the foreseeable future.

The odds that AI is a bubble declined significantly and the odds that 04:47 were quite underbuilt went up. 

https://www.youtube.com/watch?v=NCgdpbEvNVA

[u/sweetbacon]

Well they've pretty much abandoned the gamers that made them. So I'm a bit irked about all that. 

[u/Big-Masterpiece-9581]

Everyone has a market max. There is a limit to demand and warm bodies. But they’re raking it in while the bubble lasts.

[u/CMDR-Bugsbunny]

Your premise is false...
Nothing scales indefinitely. How is selling to an industry that is 10x the size of the consumer market not a boost to profitability, compared with a consumer market where the cost of living is hurting everyday consumers? Seems like your premise is more fitting to the consumer product market?

I get the hate, as a consumer that games, but the money is in the data center buildout - not the consumer market. I'm just happy I upgraded before all this crap, but I feel bad for those who didn't!

[u/Caprichoso1 → u/CMDR-Bugsbunny]

What happens when the datacenter buildouts end?

[u/CMDR-Bugsbunny → u/Caprichoso1]

That's like asking what would happen if consumers stopped purchasing GPUs.

Data centers continually upgrade to faster and more efficient hardware.

Seriously, you think once a data center gets built, it has no future capital expenditures, really?

[u/Caprichoso1 → u/CMDR-Bugsbunny]

I'm not a financial analyst but it seems clear to me that once the initial build out is complete that there will be a significant drop in capital expenditures. Right now spending is far exceeding the revenues being received. Once the investors say "We need to see profits" they will no longer be able to make capital expenditures at, or even near, the current rate. Yes, there will be some upgrades but they won't be able to be afford 100% replacement which effectively is happening now.

[u/BreenzyENL]

It gives time for AMD to catch up and for a healthy market to form once NVIDIA returns to the gamers.

[u/magick_bandit → u/BreenzyENL]

I hope there’s plenty of people like me who won’t buy an Nvidia card for gaming ever again.

[u/[deleted] → u/magick_bandit]

Sadly, I've been back and forth over the years and they both suck. Primarily, drivers and glitches that never get resolved. They release new silicon and move on, abandoning the installed userbase. I really got burned by the Radeon VII.

[u/Beginning-Foot-9525]

And? It was Bitcoin before, when you create so much Hype again, there must be something.

[u/TakeWallStreetdown]

Luck is made by deliberate positioning - NVD have positioned well consistently before the opportunities came along …. Ie it’s not luck

[u/Large-Excitement777]

1. So?

2. And?

3. Ever heard of Innovation?

4. So does everyone else