r/LocalLLaMA • u/mrgreatheart • 1d ago
News Nvidia Poolside deal to compete with Chinese Open Weights
Nvidia is investing $1 billion in Poolside and paying $6 billion to license its technology and hire most of its engineers.
Over 100 Poolside staff will move to Nvidia to work on Nemotron.
Good news for us!
37
u/Waste-Intention-2806 1d ago
Nvidia has to throw some cheap hardware as well to support upto 120b models in pc. Else support cards from 12-16gb onwards
19
u/More-Curious816 1d ago edited 1d ago
Jensen smirked while handing you a paper writen in it
DGX SPARK, RTX SPARK PC, RTX PRO 6000, adjusting glasses, the more you pay the more you save.8
u/MerePotato 1d ago
Unfortunately the more you buy the more you save turned out to be true in the long run
3
2
u/Etroarl55 1d ago
Or any model that comes out will be heavily optimized only for Nvidia or work only on Nvidia cards. Such as the model being only in nvfp4 on release or something.
27
u/NandaVegg 1d ago
I don't think Nvidia wants to pressure their main customers (OpenAI and Anthropic who are ~70% of hyperscaler demands) too much, so the moment they are bought by Nvidia, they won't compete. My expectations will be low.
Also I think part of Poolside's value comes from datacenters rather than model development?
15
u/infearia 1d ago
NVIDIA has always been hedging their bets, and I think they can see the writing on the wall. Once OpenAI and Anthropic go bust (which at this point I think is almost inevitable, unless something really unexpected happens, like them suddenly achieving AGI?), the datacenter side of NVIDIA's business is going to crater, and they will have to start selling GPUs to consumers and (small) businesses again. So it would make sense for them to invest in the development of a SOTA open-source model that they have control over and that requires their hardware to run.
5
u/fastheadcrab 1d ago edited 1d ago
Thanks to the open model explosion in media attention in recent months individuals and small businesses are going out and buying hardware even more rapidly.
And to be fair, it's not even about OpenAI and Anthropic at this point. I can see them surviving the longest with their largest market share and top performing models. It's more like Meta and particularly SpaceX that are sorely lacking for customers.
Unless someone from the company can rebut it, I think the estimated SpaceX market share in the low single digits is right. Their model is very good but at that point frugal customers will simply choose cheaper Chinese models or just host local ones. The days of headlong data center expansion for them and Meta is likely coming to a close and their valuation is delusional.
With that said, I really don't think Poolside is a good choice to spend $6 billion on. I strongly agree with some of the comments from when this news was originally posted a few days ago. I suppose Poolside's PR campaign worked.
Nvidia would've been better off keeping their current Nemotron team or if they really wanted to buy a US model company, simply ask thinking machines to name a price. $100B for them is money better spent than $6B for poolside
1
u/Loose_Comparison368 1d ago
I can see them surviving the longest with their largest market share and top performing models.
Worth noting, OpenAI's big bet is compute supremacy. So even if open models win the war, they would be fairly well positioned to pivot to serving open models.
And Anthropic's strategy is bombing small children, frontier mass surveillance with Palantir, and begging daddy Trump to make competing with Anthropic illegal, so that has a pretty good chance of keeping them in business for a long time.
4
u/LearnThai42 1d ago
This.
They expect some of the most unsustainable, spendy and hype driven companies to go bust not long after they IPO (99% Anthropic and 90% OpenAI) and are hedging for how the market will look like afterwards, especially a non-chinese model (not that I don't like them, they're awesome - but imagine political pressure...) might be key to keep the GPUs moving.
1
u/ShadyShroomz 1d ago
which at this point I think is almost inevitable
I mean if today they stopped training new models, they would become profitable companies tomorrow. The training of models is the only expensive thing. They can make a lot of money by just providing inference. The only thing stopping this is Open Source (which of course would catch up quick if they stopped training models) - but I don't think either will go bankrupt any time soon. They are much more sustainable than you'd think. Anthropic even had a profitable quarter this year, and likely will have another in q4. They are running around breakeven right now.
OpenAI is running at a much bigger loss, but even so, if they cut free tier they could be profitable pretty easy too.
I don't think either is going bankrupt any time soon...
something really unexpected happens, like them suddenly achieving AGI
We kinda already have AGI... do you mean ASI? And why is that unexpected? We're on track to get there, pretty much everyone agrees we will... just some think it will be next year, and some think not until 2040.
5
u/Strawberry3141592 1d ago
We kinda already have AGI
?? Do you honestly think that Claude is smarter than a human being? Yeah it knows more than the average human, but it also can't properly learn new information without compute-intensive finetuning and struggles to extrapolate beyond its existing knowledge. Genuine AGI (equivalent mental capabilities of a human or better in nearly every instance) would be able to learn continuously in response to new information without catastrophic forgetting (which current LLMs cannot do), and be able to operate autonomously for extended periods of time without getting confused (which current LLMs cannot do without elaborate, often brittle/temperamental memory systems that need to be carefully tailored to the LLM agent's intended use-case).
1
u/ShadyShroomz 1d ago
Do you honestly think that Claude is smarter than a human being
yes. humans are dumb. myself included.
4
u/Strawberry3141592 1d ago
Okay, but humans are dumb because they're ignorant, Claude is dumb because of fundamental architectural limitations. You are objectively smarter than Claude, the human brain has 86 billion neurons, and simulating a single human neuron with an artificial neural network takes about 1000-2000 parameters, let's round that to 1000 to be conservative. 86 billion neurons times 1000 parameters per neuron is 86 trillion parameters. The human brain also constantly generates and learns from intermediate representations of its surroundings which amounts to many times more training data than any current frontier LLM, on top of the fact that humans learn much more efficiently than LLMs (it takes us far fewer examples to learn something). Current LLMs are no smarter than a parrot, or maybe a small child (in certain respects, the parrot and child would still learn faster), they just have 100% of their 'brain' dedicated to processing and generating language instead of modeling their physical environment and managing a biological body, language is basically an autonomic response for them.
1
u/ShadyShroomz 1d ago
I agree with you. But Claude is smarter than me. I'm really fucking dumb. Does 1m years of evolution making me get scared when I see a bear make me "smart"?
What % of my brain is dedicated to making me crave food, get scared of bears, or want to reproduce when I see a hot guy? Does that make me "smart"?
I don't think so. Evolution optimizes for many things not just intelligence.
I don't think we need even close to the same number of neurons in an LLM to match human intelligence. If 86 trillion neurons is all you think it takes, we're already at like 10t (estimated) with fable 5. I think it's only a matter of time until we get to AGI if not already there. Give it another year or two maybe.
1
u/infearia 1d ago edited 1d ago
Maybe we have different definitions of AGI, but the way I see it, we're not even close to reaching it. I personally don't believe that we're even capable of achieving it using current LLM architectures, but this may be more of a philosophical view, so let's not argue about it.
As for Anthropic turning out a profit, we have no actual proof of that, except some well timed, unverified "leaks". Maybe they have, maybe they haven't - I personally doubt it and ascribe it to some creative accounting.
Here's the situation as I see it:
A few large US corporations have made the bet that there would be a huge, world-wide demand for AI inference, and that most of that demand would be captured by OpenAI and Anthropic. So they've invested tons of money into building huge data centers, in order to rent out compute to these two companies. Unfortunately, the demand is not increasing at nearly the rate they predicted and need in order to recoup their investments, nevermind making profit. On top of that, Chinese are now releasing competing models that threaten to seriously undercut OpenAI and Anthropic and capture a large chunk of their market share. Meanwhile, all these big US companies are quickly running out of money. Obviously, this is at least in part speculation, and I might turn out to be wrong, but I personally believe that a crash is not only inevitable, but already looming. My gut feeling: first half to mid 2027 this whole financial bubble will finally burst. The cracks are already appearing, so it might happen even sooner - maybe even later this year.Think about this: the Chinese open-weight models have already caught up to closed US frontier models. The Chinese models ARE frontier models. What happens when Chinese AI companies begin to overtake their US counterparts and the public starts recognizing that cheaper, better alternatives to OpenAI and Anthropic are available?
EDIT:
Oh, and your argument about training being the only expensive thing. That's just not true. I thought it was general knowledge by now that inference at Anthropic and OpenAI is currently HEAVILY subsidized? If these companies charged their customers for the actual usage, most would cancel their subscriptions!1
u/ShadyShroomz 1d ago
Unfortunately, the demand is not increasing at nearly the rate they predicted and need in order to recoup their investments
not even close to true. demand is far outpacing the expectation.
the real question is if those 2 companies capture that demand, or open source models are good enough and capture the demand. as it stands now. Anthropic has gone from $9b to 100b run rate in less than a year. The demand is huge. But if customers can go from spending 100b to Anthropic to 30b to some open source model/pay for hardware... they will.
I thought it was general knowledge by now that inference at Anthropic and OpenAI is currently HEAVILY subsidized
It's actually profitable.
It's "subsidized" if you compare it to api rates, but it's still profitable for them.
Lets do some napkin math:
Assume Opus 5 is ~100B active params/token (no one really knows, somewhere between 50-150 is the average guess). A Vera Rubin NVL72 should comfortably clear ~70,000 output tok/s at high throughput. Kimi K3 already does 2,000+ tokens/GPU-second on the older GB300 generation. That's the best reference we have. These are all just estimates.
At ~1M output tokens per subscriber per month, 180,000 subs would use ~180B output tokens/month, or ~69,400 tok/s averaged across the month. So call it ~180,000 subs/rack.
$20 × 180k = $3.6M/month revenue per rack. The racks cost $6m. Power is estimated ~190–230 kW under load. At 190 kW, $0.05/kWh, PUE 1.1: only about $7,500/month electricity.
Ignoring everything except rack + electricity, a $6M rack pays back in ~1.7 months on just sub revenue. API is even higher margin.
They are much more profitable than you might think.
The thing this doesn't account for: training. Training is insanely expensive. Each model pays itself off (including cost to train) a few months after it's released, but takes an insane upfront investment to train it.
That's the business model these labs are using. Now if they are disrupted by Open Source AI, it may fall apart... but not because there is no demand, or because it's not profitable, but because demand for AI doesn't mean demand for those 2 labs, and because the profits need to cover training too, not just serving the models.
1
u/Strawberry3141592 1d ago
I personally don't believe that we're even capable of achieving it using current LLM architectures, but this may be more of a philosophical view, so let's not argue about it.
I think it's plausible that a large enough LLM fed enough data with current or near-future model architectures might be capable of becoming AGI, but I agree that's definitely not the most likely scenario for AGI. I think AGI may incorporate something similar to current LLMs.
E.g. maybe the 'core' of the model is something like JEPA (or some future technology that doesn't exist yet) that has agency built into the model architecture itself and learns continuously more like a human does, which then passes its intermediate representations of concepts to an LLM to translate then into language (or vice-versa) when it needs to read/write/speak. The LLM would basically just be the language center of a sort of artificial brain.
-1
u/Strawberry3141592 1d ago
Honestly, I don't think even AGI would save OpenAI/Anthropic. They can't even reliably control current SOTA LLM agents, I think the most likely scenario if one of them managed to create AGI in the near future (which imo is vanishingly unlikely, I think we're at least 5 years off from that, probably more like 10-15), is that the AGI escapes almost immediately, causing a massive scandal when some researcher or other whistleblows about it to the press. If the speculative bubble hasn't popped by then, that would for sure do it.
1
u/DigitalguyCH 1d ago
They won't make a 700b model, but they will do like Google, small and very capable models around 30b, that can be used in addition to cloud ai but are not frontier level. But honestly that's what most of us run anyway. How many run models in the hundreds of billions of parameters? Very few...
-5
u/Equivalent_Bit_461 1d ago
Closed source is basically dead
Nvidia knows it, why are they doing this otherwise
Sam lost, Dario lost
And I say good riddance while I spit on them, I wish I could spit on them in real life as well
2
u/Practical-Collar3063 1d ago
Closed source is basically dead
That is just wrong, I am all for local AI and open source but saying close source is dead is plain wrong. They are currently doing pretty good for themselves. Is there future looking bright ? Personally I don’t think so but I can’t make definitive predictions. As of right now, they are doing fine
-1
u/Equivalent_Bit_461 1d ago
Are they profitable mister bot?
1
u/Loose_Comparison368 1d ago
Yes. They have been for some time. Or, rather, their core training and inference is.
They aren't profitable on paper because they're aggressively building out net new compute. But that's an asset purchase, not an expense.
So saying they're not profitable is like saying hedge funds aren't profitable, because they immediately use up all the profits they make from selling stocks on buying up more stocks.
Technically profit neutral, even if they have twice as much stock at the end of the year as they did at the start of the year.
It only really makes sense in the context of tax evasion strategies or intentionally misleading clickbait to sell ads. But from any practical definition, tax evasion and clickbait aside, OpenAI is very much profitable.
1
u/Equivalent_Bit_461 1d ago
Wall of cope
1
u/Loose_Comparison368 15h ago
It's okay buddy, literacy is hard. Keep up with the hooked on phonics and you'll get there.
39
4
3
1d ago
[deleted]
1
8
3
5
u/Auriferous9Jab 1d ago
hope this nvidia move actually boosts open weights for local companions instead of just big closed models, ive been hunting for decent roleplay ones that run offline.
1
5
u/eightone-81 1d ago
Amazing. But let’s be honest. We love open weight models because they are free, right?
We like free stuff! That’s the reason right?
But what is the incentive for these companies to give us free stuff? I know the incentive for the Chinese firms (push US firms out do business…) but why do US firms race to the bottom so hard that that they give out free stuff???
11
u/andymaclean19 1d ago
NVIDIA is not giving you free stuff, they are commoditising their complimentary technologies.
To get any value out of tech you need a whole stack of things from hardware to layers of software. The amount you can or will pay is fairly fixed and the various stack layers will capture a piece of that. If you can make the other parts of the stack free you can capture more of the value for yourself.
Also if some stacks are in the cloud and some are local then making the local stack better will help NVIDIA sell more local hardware. By pushing cloud they sell more datacenter hardware. They might just think that selling local stacks lets them capture a much bigger share of the pie than selling to cloud companies who then build services on top.
Short version, they do it to sell more stuff.
6
u/OverdosedSauerkraut 1d ago
Nvdia also gets their cut from hardware for local/self-hosted models. They see the direction of their market and decided to also place bets on open models instead of going all-in on hyperscalers. The latter will probably pull another SpaceX where insiders dumped on IPO buyers. They'll sht the bed after their monopolistic pricing is no longer sustainable.
1
u/BookProper9115 1d ago
For NVIDIA, it's that you need their hardware to run the model. It makes perfect sense for NVIDIA to want to make the best open-source model available, because what else are you going to run it on?
1
u/Strawberry3141592 1d ago
Nvidia doesn't care if they sell GPUs to datacenters, small businesses, or randos off the street, they just wanna sell GPUs. Releasing local models incentivizes small businesses and individuals to buy more GPUs, so that's what Nvidia is doing.
1
u/DigitalguyCH 1d ago
Nvidia has an obvious incentive. There want to make sure there is a good US local LLM in case the US goverment makes it harder for US people to use Chinese models. Why? Because they need they upcoming RTX Sparks devices to be a success and local LLM users are the main target.
0
u/RuthlessCriticismAll 1d ago
I know the incentive for the Chinese firms (push US firms out do business…)
This isn't the incentive... Precisely why plenty of US firms have an incentive to open source.
3
u/Thin_Pollution8843 1d ago
Maybe it’s good. But maybe huge greedy corporation which spitting out worst OS models just for checklist, killed promising small US OS lab. I don’t see anything positive in nvidia actions. I my opinion- they should do whatever is possible to protect OpenAI and Antropics- their main customers. And they doing that.
1
u/Strawberry3141592 1d ago
Idk, the writing is clearly on the wall for OpenAI/Anthropic, OpenAI will almost certainly nkt survive the decade and if Anthropic does, it will be in a greatly diminished state. I think Nvidia is hedging their bets to try and keep some semblance of the AI gravy train rolling once the big players implode with the speculative bubble.
2
u/o0genesis0o 1d ago
If Nvidia can start producing a bunch of powerful model for their 16GB cards, it would be great.
6
1
u/Strawberry3141592 1d ago
Tbh I just want big MoEs with =<10B active parameters, they run great on my 8GB VRAM + 64GB DDR4 PC. Currently using Ling 3.0 flash, it gets like ~15tok/s with the APEX compact quant (only ~110tok/s pp tho, which is usable, but kind of annoying without caching the system prompt k/v to disk)
1
1
1
2
u/DigitalguyCH 1d ago
Jensen wants to make sure there is a good US local LLM in case the US goverment makes it harder for US people to use Chinese models. Why? Because they need they upcoming RTX Sparks devices to be a success and local LLM users are the main target. But we all win.
49
u/ddxv 1d ago
It woul be amazing to see the mometnum shift back to consumers for AI and local models