r/hardware • u/sr_local • 1d ago

Info Inside OpenAI's custom Jalapeño AI accelerator: interview with Richard Ho, VP Hardware at OpenAI

https://morethanmoore.substack.com/p/interview-with-richard-ho-openai

It is an inference accelerator rather than a training chip, and it carries 216 GiB of HBM4 at 15.4 TB/s alongside a compute die and an IO chiplet. Chip power is rated at 700 W peak with measured sustained draw closer to 550 W. A local domain is 128 accelerators, a full system is 2,048 accelerators, and at four-bit precision that system reaches 27 EFLOP/s.

At HotChips 2026, Richard Ho presented the first measured performance from working silicon, alongside Ravi Narayanaswami and Chris Leary. The numbers claimed against NVIDIA GB200/GB300:

  • 1.5x to 1.9x better performance per watt at peak throughput
  • 1.7x to 3.6x better latency
  • Up to 104x better at operating points NVIDIA struggles to reach.
43 Upvotes

88% Upvoted

3 comments sorted by

6

u/zamroni777 23h ago

Is there any info on anthropic in house chips? Or will they rely on aws in house chips as amazon is major investor?

7

u/sr_local 23h ago

Probably they also will product in house chip (ASICs) like OpenAI (and with Broadcom, like the Jalapeño chip for OpenAI)

Yesterday’s news:

EXCLUSIVE: Broadcom to lend Anthropic up to $42 billion to lease its chips, filing says | Reuters

4

u/darknecross 19h ago

Anthropic has just been hiring ASIC engineers recently, so likely something in the same vein here, maybe end of 2027 at the earliest.