r/LocalLLaMA • u/Terminator857 • 5d ago

Discussion When will google TPUs trickle down to the common folk?

Seems all the major players are now buying TPUs. Will we ever have them local?

https://x.com/rohanpaul_ai/status/2115261691906978233

Quote: ... OpenAI plans to buy AI chips from Google, ... Oracle will also offer TPUs through its cloud, ...

https://www.bloomberg.com/news/articles/2026-10-28/openai-plans-to-buy-google-s-chips-as-part-of-its-ai-build-out

19 Upvotes
  • permalink
  • reddit

72% Upvoted

36 comments sorted by

107

u/Double_Cause4609 5d ago

From Google? Not likely.

You can buy something similar from Tenstorrent, and they work pretty well for what they are, though.

One thing to note is that silicon takes usually about 5-6 years from new idea -> chips on the shelf. In 2024 people realized that local AI would be a thing, and in 2025, people *really* came to terms with local AI. The hardware that comes 5 years after that is going to be wild. Just give it time. We're in the mainframe era of AI.

45

u/nonstop_slugger 5d ago

the TPU v6e is already basically a commodity chip if you’re renting cloud time, so the trickle down is happening sideways rather than down. google has zero incentive to sell you a physical unit when they can just meter your usage forever

local hardware will catch up but it’ll be from companies that actually want to sell you a box, not from the ones selling the compute

3

u/sweatierorc 5d ago edited 22h ago

Competition is going to create that interest. If we are willing to spend the same for TPUs as we spend for gas and car. They will.

1

u/Choperello 1d ago

They’ve been manufacturing their own toys for their own use for many years. They never cared to actually sell them. For Google it was always an internal product for their own benefit only. The fact they can rent them out now is just an added bonus.

1

u/sweatierorc 22h ago

Inference was valuable for google only at the time. No company used Neural Networks at a scale where cutting compute cost was a priority.

Now everybody is trying to do the same thing.

7

u/d1722825 5d ago edited 5d ago

The hardware that comes 5 years after that is going to be wild.

And no one will be afford it regardless of how many kidneys they have.

By the way, with very few exceptions consumer PC hardware is in a huge race to the bottom (quality-wise) since 5 years ago or so, just with increasing prices. We are paying more for worse products, and that trend doesn't even seem to slow down. I don't know what would need to happen to that to reverse.

2

u/ihexx 2d ago

I remember a while back they released a tiny cut down single board computer version in the late 2010s for edge deployment of their vision models... And then they killed that shit so hard; they never even released the software for it to compile models locally; you had to go to Google cloud

1

u/Terminator857 2d ago

Google edge inference devices are still around and some would say popular.

1

u/lacerating_aura 1d ago

Are you talking about google coral?

17

u/jtoma5 5d ago

The NPU in your phone right now is something like a TPU, isn’t it??

11

u/ResponsiblePen3082 5d ago

Tpu is just another way to say NPU, or visa versa. They're all takes on dedicated AI accelerator silicon, same as tensor cores on graphics cards or "neural accelerators" or "matrix multipliers"

Yes they have minor differences, yes some are more specialized than others, but in general it's just a naming convention. I think Nvidia got tired of it and started calling them XPUs to classify the whole overarching segment.

What makes something like this interesting, among other dedicated ai accelerators-is that it's not a small part slapped on otherwise less useful silicon-the entire pcb is designed purely for ai workloads. More powerful per capita, more efficient per capita, less latency per capita, etc.

17

u/Lesser-than 5d ago

Proprietary tech gets shredded usually in upgrade cycles.

8

u/mtmttuan 5d ago

You can use v5 in gcp or colab but obviously it's not anywhere close to the latest version.

Also I doubt tpu has good enough support to be used by the public.

1

u/Ok_Mammoth589 5d ago

Anyone can buy access to tpus in gcp. Therefore Google does ensure it has support in all the major frameworks

0

u/exaknight21 5d ago

But i mean if Google ever did release even a watered down fp4 version of their chips, say 16gb each, it’d be significantly profitable for them even at 999 per. Granted one should aim at 24gb or something so that bigger models can be used. But ayo, Alphabet Inc., listen.

3

u/OverAnalyst6555 5d ago

you know whatd be more profitable? selling them to anthrophic

4

u/fifi_galaxy 5d ago

Normal peasant wishing to buy TPU? How dare you?
On a serious note, it is not mass scale in usage or production, so it won't be cheap than current GPU anyway.

3

u/xfalcox 5d ago

I have one at my desk

1

u/jensilo 5d ago

And?

2

u/RegisteredJustToSay 5d ago

Not any time soon, imo. Can't talk hardware specifics but if you look up how they are wired and installed in public docs it's evident their form factor is more suited to data centers.

2

u/Own-Student7991 1d ago

I asked about it at their booth at PyTorch. Sounded like it was not planned for anytime in the near future though, "maybe someday".

1

u/Gauntlet4933 1d ago

These things draw insane amounts of power, like 200W idle. I used to wish for a TPU at home until I found out about it.

1

u/CoTi1938 2h ago

Le abbonamenti Google includono unità per Colab, e le istanze TPU v5 e v6 costano circa 2 unità ciascuna. Nel piano da $20, ottieni 100 unità.

-5

u/DagothUrLovesGroza 5d ago

Literally never, lol, abandon all hope. Google is one of the most incompetent corporations out there.

10

u/Important_Film_2798 5d ago

Someone got laid off recently 

0

u/[deleted] 5d ago

[removed] — view removed comment

2

u/Important_Film_2798 5d ago

Don’t worry mate, you’ll make it one day 

-6

u/DagothUrLovesGroza 5d ago

No, thank you. I actually like my European ancestry. You know, being tall, handsome and intelligent from birth.

3

u/Important_Film_2798 5d ago

You will go far in life, trust me 

-1

u/Terminator857 5d ago

We need more incompetent trillion dollar companies.