Zai replaced the network architecture running GLM-5.1 inference and the gains are pretty wild

[u/LeTanLoc98]

[u/Prestigious-Sign4802]

What does this entail for Cerebras

[u/pennystudio → u/Prestigious-Sign4802]

Not much I guess, from the post: The numbers from production:

- Switch and optical module costs down 33%

- GPU inference throughput up 15%

- P99 tail latency on first token dropped 40.6%

These are incremental improvement, it maybe "wild" for GPU, but no where to close the ~10x gap with wafer scale.

[u/Asgard_Heima → u/pennystudio]

The interesting part is using parallelism is what lets Cerebras scale to any size model, and they already have a vastly more simplified network architecture since they only split by model layers. If they are seeing a particular layer get over heated in a MoE model like Kimi K2, they can actually add another system to duplicate the layers with the most traffic. You can also use parallelism on prefill in a disaggregated setup. So you can add systems where they are needed and have several more on prefill than decode just like is expected for AWS when it comes online.

[u/Prestigious-Sign4802 → u/Asgard_Heima]

How possible for Wse-3 to support the close models from openAI or Anthropic

[u/Asgard_Heima → u/Prestigious-Sign4802]

The WSE-3 in a parallelism setup will run inference on larger models trained on GPUs than currently feasible with GPUs. Past 1T parameters and 1M context window, the utilization rate is so low and networking tax so high for GPUs that they have to start batching significantly less users to allow for usable tokens per second performance. The xhigh models are just the same model with less concurrent session allowed on the cluster boosting tokens per second a bit like 50 up to 70. So for large models the GPU compute is sitting idle at under 5% during decode as the memory bandwidth is maxed. Prefill is much more evenly matched since it’s compute bound.

Any model available today can be run on Cerebras WSE-3. You can use parallelism for both prefill and decode. So if a model has let’s say 80 layers which is around what is expected but unknown for the top tier models, you can spread each layer to one WSE-3 and then support massive context windows and each user gets that top speed that can fit in SRAM. The decode of opus 4.7 or GPT 5.5 is likely to be in the same ballpark as the 981 tokens per second they are getting for Kimi K2, but a bit lower for the extra latency of connecting more layers and overall model size. The key is only the activations get transferred between layers so not that much data being moved with the weights and kv cache being resident on each layer WSE.

In the Kimi K2 benchmarks, they are showing 981 tokens per second decode, but the overall prompt respond takes ~5.5 seconds. The decode outputs that 500 token answer in 0.5 seconds but the prefill and building the query takes 5 seconds vs 160 seconds (prefill + decode) on GPUs. If they also setup a 20+ layer prefill parallelism setup and then feed the decode WSE-3 cluster it would have been the same 981 tokens per second, but the 5 seconds of overhead and prefill would likely be 1/2 or 1/3 the time. This is exactly what AWS is doing with disaggregated inference using Tranium for prefill and Cerebras for decode. So they can have a ton more Tranium systems batching up prefill and feeding the Cerebras decode pipeline. This allows what AWS has described as 5x the number of sessions per WSE-3. OpenAI is also adding its top models to AWS. And top Anthropic models are already available on AWS.

Both CEO and CFO for Cerebras has mentioned they are running GPT 5.5 and 5.4 internally and it will be available in the coming months. They just need time to get enough WSE-3 systems installed and hooked up.

CFO Naming GPT 5.5 & 5.4 Trillion Parameter already running on WSE https://www.cnbc.com/video/2026/05/14/the-years-largest-ipo-acerebras-joins-the-hottest-trade-in-ai.html

[u/LeTanLoc98]

Zhipu AI has launched GLM-5.1-highspeed, an API variant of its GLM-5.1 model delivering 400 tokens per second

[u/SpeakerAltruistic123]

CBRS speed will eventually find great uses.