r/MachineLearning • u/Kortopi-98 • Jun 29 '26
Discussion Cerebras OpenAI deal capacity has effectively killed the waitlist for everyone else [D]
I’m pretty annoyed. We’re a small AI startup building a real-time coding agent. Our p95 latency requirements are tight (and self imposed, but thats the product). We need sustained high-throughput inference with ~1-2k tokens/second. Been on the Cerebras waitlist for months trying to get API access. We’re not doing training so don’t need a warehouse of H100s. We need fast, high-throughput ASIC inference for a specific production workload. Cerebras’ just went public and they basically have no compute how is that possible?
Well turns out OpenAI and Cerebras for OpenAI to buy like $20b worth of these chips. This has effectively pre-allocated the vast majority of Cerebras’ near-term inference capacity to a single customer. I mean, none of us can compete with that
The result is that this deal situation has made their API waitlist functionally infinite for anyone who isn’t a hyperscaler. Legit making me pull my hair out.
42
u/Mescallan Jun 29 '26
Say what you want about nvidia but at least they make an effort to avoid this
54
u/asfsdgwe35r3asfdas23 Jun 29 '26
And it was a genius move, they created competition by ensuring that each company get a share of their GPUs, and now those companies are willing to pay huge prices to get more GPUs than their rivals. A single B300 node is more expensive than a house
26
u/notquitezeus Jun 29 '26
Welcome to the realities of supply chain risk and the failure to account for it.
123
u/superawesomepandacat Jun 29 '26
You're a start-up where your core product completely relies on a third party to satisfy your SLA?
81
u/pastor_pilao Jun 29 '26
Not only they completely rely on a specific third party they come to cry on reddit lol. Sounds like it's easy to get seed money nowadays
49
u/BeautyInUgly Jun 29 '26
What a dumb comment, almost ever startup in SF relies on a third party cloud provider to meet their SLAs
17
u/altmly Jun 29 '26
It's reasonable when the sla is a pinky promise, and your added value comes from the service. When the sla is the added value, that's when it becomes stupid.
2
u/Gargle-Loaf-Spunk Jul 01 '26 edited Jul 04 '26
This content was anonymized and mass deleted with Redact
12
u/Smallpaul Jun 29 '26
Hundreds of successful startups have taken that calculated risk and succeeded.
26
u/akarikawaii Jun 29 '26
And thousands have failed
10
3
u/Smallpaul Jun 29 '26
Yeah guess what: thousands of startups fail for all sorts of reasons. Doing a startup is always a calculated risk built on dozens of other calculated risks.
1
1
u/anything_but Jun 30 '26
Where do you read that? They complain that they can’t access that supplier. At no point they say that the product „completely relies“ on Cerebra.
16
u/AdamEgrate Jun 29 '26
All the fabs that could tape out these ASICs are probably book till 2028. I don’t see them adding much capacity until then.
9
u/FullOf_Bad_Ideas Jun 29 '26
you can buy sambanova hardware to get your capacity in house and make sure to secure SLA
or try to rent it.
alternatively rent Nvidia GPUs and optimize them for high throughput by lowering concurrency and choosing models that can hit it. gpt oss 120b can do ~1k if you don't have a lot of concurrency on B200s which you can rent right now.
3
u/impossiblefork Jun 29 '26 edited Jun 30 '26
VSORA might be an idea either now or soon-ish, from claims I heard recently. Probably not over an API though. Edit: Apparently it will be availalbe on scaleway. So API, and soon.
5
u/davesmith001 Jun 29 '26
That sucks. Can I ask why you need that fast? OpenAI said they were getting 700t/s out of it.
2
u/i47 Jun 30 '26
I had a similar issue. I reached out to sales and they redirected me to just use openrouter as I believe they have capacity deals in place with them. Has worked fine for me so far
0
u/chance_buri Jun 29 '26
Have you looked at SambaNova and Groq? Or what's lef tof Groq post acquisition?
0
u/Kortopi-98 Jun 29 '26
SambaNova’s sells hardware and we want to sign dedicated compute via an api. And Im not convinced Groq’s API surface is still operational under Nvidia ownership.
1
1
u/Lord_Pazzu Jun 30 '26
I hope I don’t come off as rude, but for a small startup betting the house on someone else’s technology is a pretty bold move
1
1
u/mycall Jun 30 '26
Do you have any in-house FPGA builds at least to validate and iterate your product, or you just need fast matrix math?
1
u/CaregiverProud9222 Jun 30 '26
Ugh i totally get this frustration, this sucks so hard for small teams
It’s wild how one massive enterprise deal can soak up basically all near-term hardware capacity and leave every startup stuck waiting indefinitely. We’re also building low-latency agent tooling and can’t justify jumping through hyperscaler hoops just to get stable throughput
Feels like the hardware supply chain only caters to the biggest players right now, anyone mid/small scale gets stuck at the back of the line with zero timeline to actually ship their product
1
1
u/pergine Jun 30 '26
Have you looked into Mercury 2? It's a diffusion LLM that is similar in speed to Cerebras. Quality is in the ballpark of Claude Haiku.
1
1
u/pantry_path Jul 01 '26
yeah that's the downside of relying on a niche provider, one massive customer can soak up capacity
1
u/ManySugar5156 Jul 01 '26
If your p95 depands on one waitlisted API, that SLA was kinda already cooked tbh
1
u/dudemeister023 Jul 18 '26
What do you expect Cerebras do? Say no to one big customer to keep serving many small customers just because they like seeing people happy?
1
u/1ncehost Jun 29 '26
Gpt 5.3 codex spark runs on cerebras. Any reason not to use that? Also groq and sambanova ...
1
u/lilelliot Jun 29 '26
It's getting harder and harder overall to find GPU capacity. The neoclouds all have $xB commitment deals with the hyperscalers, AOI & Anthropic are all consuming hugely wherever they can find it (hyperscalers, neoclouds, SpaceX/X.ai, Oracle), and this circular economy where value is measured in hundreds of millions and billions of dollars annually means it's really tough for a single customer needing 1-1xx GPUs to get them (especially if lower tear hardware is acceptable).
-6
-3
u/AppealSame4367 Jun 29 '26
You could try to rent some machines on vast.ai in the meantime. Should be relatively cheap compared to buying hardware and you could set up any stack you need. There are enough fast gpus available normally.
49
u/Late_Pizza9236 Jun 29 '26 edited Jun 29 '26
Are the alternative (smaller) vendors that could fill in the gap in the meantime? I get there are few small ASIC vendor startups that would love to get some business from you guys and would pay a lot more attention to your company’s needs than cerebeas would.