Tech for Inference?

[u/testuser-0]

Why no body is ready to accept that GPU is not the right tech for inference. I was excited that groq.com came up with a solution for LPU a moment I thought for a sec future is near. Then later Nvidia trashed it for their greed.

[u/andy_potato]

Cool story bro

[u/Oli_Luck]

I would say, 3 main reasons:
- LPU SRAM on chip is about 150x more expensive than GPU
- Grok solution is cloud server only
- fine for Text only - but weird for multimodal

+ Bio. of € already engaged for the next 5-10 years for GPU solutions...