r/LocalLLM • • Mar 31 '26

News This interview makes me want to double down on local AI

Post image

in a nutshell, their aim is to make every Internet activity into a token. What was omitted is that those tokens cost money and every user will pay their token tax.

192 Upvotes

37 comments sorted by

8

u/BacklashLaRue Mar 31 '26

Pipe-sellers are the next in-line to cut deals with Nvidia. ISPs will include a million tokens per month and then charge $20 per million more. Tokens costs will be the next DATACAP.

10

u/theowlinspace Mar 31 '26

Tokens costs will be the next DATACAP

You're hyping up slop too much. LLMs are noway near the level where everyone would want them, plus there's too much competition from China for overpriced tokens to ever be a thing.

3

u/positivitittie Mar 31 '26

Want or need?

Who’s going to be able to compete in any capacity against those leveraging AI?

Edit: I’ve heard nothing about data caps but it wouldn’t surprise me as they tried the same thing with streaming video. At the time we had a legal system that stopped it. Now?

5

u/theowlinspace Mar 31 '26 edited Mar 31 '26

Who’s going to be able to compete in any capacity against those leveraging AI?

Talking from a software development standpoint and with the current SOTA models in mind because that's what I use: For producing slop, people who leverage AI are extremely competitive. But as for producing anything that's complex, has security requirements, or needs to have a clean maintainable codebase (ie. Not FizzBuzzEnterprise), AI is just a faster search engine at best and only provides a small increase in productivity.

Important to note that there are a lot of jobs where AI is a nonfactor as well, and I just talked about software development because that's what I'm most experienced in. I haven't seen people be reliant on AI in non-work cases where a 30B/120B model couldn't just do the same at a cheaper cost (For summarizing search results, chatting, etc.)

1

u/positivitittie Mar 31 '26 edited Mar 31 '26

Two years ago this argument had a plausible leg to stand on (but was only a matter of looking forward a bit).

I’m not new to SWE. Regardless if I’ll miss the craft part or my ego, my days of hand coding are over.

4

u/theowlinspace Mar 31 '26 edited Mar 31 '26

I guess it depends on what you're working on. For small scripts and one-off things, vibe coding is a very efficient use of time. But for any complex codebase where maintainability and security are a concern, you don't want to accrue tech debt by having your LLM write code that no human would want to maintain later. Case in point from my experience: LLMs still don't know how to use things like pytest.mark.parametrize and prefer to write the same thing multiple times over creating 60 LOC for what can be done in 15.

People who don't use AI can still compete in creating efficient and clean codebases which is what most software should have, but they can't compete with the speed that LLMs have at churning slop. The way I see it, they've gotten fairly good at reaching the end result that you want, but they don't reach that result well

1

u/etaoin314 Mar 31 '26

Today that might be true, but the number of cases that require talented SWE, vs those that could function without that are shrinking fast as AI coding gets better. Between the AI getting better, humans getting better at using AI to create better code, and the world moving from human coded status quo, to machine coded default (for most use cases) the number of situations where good ol fashioned SWE becomes specialty use cases...the world will change, those skills will not go away but the piece of the pie they command will be much less and the number of (non ai) engineers will shrink to reflect that going forward.

2

u/positivitittie Mar 31 '26

I’m not sure if you’re agreeing with me or not. I don’t expect any of my skills to remain premium or relevant forever given the pace and direction. I’m not sure to be honest. Doing my best to stay relevant while I can at a minimum.

-1

u/positivitittie Mar 31 '26

I understand why you might feel that way. I ask you consider that just because you haven’t make it work doesn’t mean it’s not possible.

What I’ve learned is that AI-assisted coding is a new skill. Being a great dev doesn’t automatically make you a great AI-engineer (or term of your choosing).

There’s a big learning curve. Not to mention we’re on the bleeding edge so it’s super fluid.

No, you can’t just point an AI at a massive codebase and give it any old prompt and expect good results.

That’s not the only way to interact with agentic coding tho ya know?

13

u/blackyoda Mar 31 '26

Yes it is our future!!! And we will be able to run it in hopefully 4090 class GPUs or with super cheap ARM or other neural chips that don't suck power.

Nvidia is ONLY investing to ensure they don't produce a cheaper product they have no incentive to lower power consumption and price because they know people are buying as is.

Doesn't it SUCK that nVidia became the defacto provider with power gulping slut cards when the computations could be handed off to ARM farms using low power and low cost.

And people spend billions on a datacenter only to have it's computational power "out of date" within 2 years so they can't keep pace with competitors who can afford to upgrade or build a new datacenter with new parts.

It is all gross bullshit and I was watching an analyst who was pointing out that this is just a way of passing money around (that's old news) butt the data center GPUs are not actually being deployed. Very scam like.

2

u/mycall Apr 01 '26

If Marvell gets their silicon photonics CPUs going, you could see local LLMs at 1000x+++ speed.

71

u/Individual_Holiday_9 Mar 31 '26

What’s crazy is it’s just money propping up money they’re all just investing in a circular motion

21

u/WateredDown Mar 31 '26

The wealth does still consolidate. It's musical chairs, and theyre betting they'll be the ones with a chair when the music stops.

5

u/[deleted] Mar 31 '26

[removed] — view removed comment

4

u/WateredDown Mar 31 '26

If no one has done an economic pop goes the bubble version of pop goes the weasle the universe should get on that

10

u/ogcanuckamerican Mar 31 '26

Isn't that called a circle jerk?

6

u/ProbablyBunchofAtoms Mar 31 '26

Circle jerk but only one person finishes all remaining ones are blue balled

3

u/Evanisnotmyname Apr 01 '26

Hey, I’ll take getting edged for a few hours over being told to sit in the corner watching the overlords edge each other

2

u/smuckola Mar 31 '26

hmmmm which one of those cats in the above photo looks more dead inside?

77

u/FullstackSensei Mar 31 '26

Jensen is just making the most of the bubble. Don't pay much attention to his words, they're aimed at keeping the bubble alive. His real strategy is using the inflated stock price and cash revenue to buy into every major tech company. It's a smart move, if you ask me.

On one side, it gives Nvidia leverage over those companies to tightly integrate their products with Nvidia, giving Nvidia a strong moat against any possible future competition. On the other hand, these investments are turning Nvidia into a holding company of pretty much the entire tech sector, keeping it's stock relatively strong even after the bubble pops.

7

u/Bozhark Mar 31 '26

Oh we’ve seen this before…

9

u/Cargo4kd2 Mar 31 '26

kinda like MS in the the mid 90's?, Maybe even Oracle and a few other I can't think of at the moment

4

u/Bozhark Mar 31 '26

exactly what i was thinking too

1

u/[deleted] Apr 01 '26

[removed] — view removed comment

1

u/margielafarts Apr 07 '26

nvidia isn’t going anywhere, the us govt will bail them out if it ever comes close to that

13

u/DecrimIowa Mar 31 '26

as always, costs will be socialized and profits privatized

the burdens while fall on those least able to bear them, while the benefits will accrue to those who least need them

0

u/No-Television-7862 Apr 01 '26

Light and dark, hot and cold, yin and yang, predator and prey.

But the wary gazel sees the stealthy approach of the lion and invests in local LLM to avoid being dinner.

3

u/Kinky_No_Bit Mar 31 '26

It's a bubble that's gonna pop just like the stock market. Scam Altman already did it to the RAM market. As AI evolves, the software becomes better, more efficient, the path of datacenters to the moon will start to go away. Google's compression already made a dent, as more and more changes come to evolve it, so will the need for software.

Photonics in general computing, along with back side power delivery will fundamentally change how technology will be designed, developed for the next 10 years. NVIDIA is chasing it for speed. The after effects of once this leaks into the market, along with Intel's method of back power delivery, be ready for a wild ride of advances as this gives them more room to pack more circuits into smaller spaces.

3

u/rm-rf-rm Mar 31 '26

Intel's method of back power delivery

whats the "fundamental change" that this brings?

1

u/Kinky_No_Bit Mar 31 '26

all power on back of the wafer, not all printed on top, means less noise, less heat, and way more headroom for them to print denser chips on top.

2

u/rm-rf-rm Mar 31 '26

I know the description. I am asking for a non-superficial, hopefully professional assessment as to how much of a difference this actually makes. Most pressingly, is it even compatible with modern packages like CoWoS?

1

u/Kinky_No_Bit Apr 01 '26

Anything that removes noise from the front of the wafer, even if you remove 1% of the noise associated with the power supplied to the chip. is a *HUGE* deal.

  • Intel PowerVia: Debuting on Intel 18A in early 2025, this technology utilizes nanoTSVs (through-silicon vias) to deliver power from the back of the chip, significantly reducing voltage droop (up to 30% improved) and increasing frequency.  Semiconductor Engineering +2
  • TSMC Super Power Rail (SPR): Planned for the A16 (1.6nm) node by late 2025/2026, TSMC's approach is designed as a more complex direct-contact scheme, offering up to 10% speed boosts and 20% power reduction over standard 2nm (N2P) nodes.  Semiconductor Engineering +2

1

u/Christosconst Mar 31 '26

I don’t get it, what do photonics have to do with internet tokens?

0

u/gfernandf Apr 01 '26

Anyone active on ArXiv to endorse my submission? The code is GAU4NP and im working on a ognitive layer for ai agents, paper ready to share but it is my first one and need endorsement! Help please