750 Tokens per Second at the Frontier: How Cerebras Wafer-Scale Powers GPT-5.6 Sol Ultrafast

Contents

01 · A New Speed Class Arrives at 750 Tokens per Second
02 · The GPU Inference Bottleneck Is Data Movement, Not Compute
03 · One Whole Wafer per Chip: The Cerebras Architecture
04 · What the Numbers Show, and What Remains to Be Verified
05 · Key Questions Ahead
06 · Sources

A New Speed Class Arrives at 750 Tokens per Second

On August 13, OpenAI and Cerebras unveiled Ultrafast mode, a new service tier running OpenAI’s GPT-5.6 Sol on Cerebras hardware to generate up to 750 output tokens per second ¹ ². That represents an acceleration of up to 14x over Standard processing, launching first in the OpenAI API as a limited preview for select customers ². Cerebras claims this throughput comes without any compromise in output quality ¹. OpenAI framed the launch as a break with a longstanding trade-off — where achieving real-time responsiveness historically meant downsizing to smaller or specialized models — shifting the paradigm toward delivering vastly more useful work per second ².

The GPU Inference Bottleneck Is Data Movement, Not Compute

Large-model inference is fundamentally throttled by data movement rather than raw compute capacity. On GPUs, massive models must be partitioned across multiple chips, forcing weights to shuttle back and forth between on-chip memory and off-chip storage with every generated token — creating a severe memory-bandwidth bottleneck ¹. As models scale up, gains in capability bring steep increases in compute and communication overhead, slowing response times and forcing users to choose between waiting for high-quality results or settling for inferior outputs sooner ¹. Cerebras takes a contrarian architectural path by eliminating this off-chip data movement altogether ¹.

One Whole Wafer per Chip: The Cerebras Architecture

The Cerebras Wafer-Scale Engine utilizes an entire silicon wafer — never cut into discrete dies — as a single monolithic processor ³. Packing 44GB of onboard SRAM per wafer, the architecture retains model weights entirely on-chip while tokens stream uninterrupted across layers pipelined across wafers ¹. Its latest-generation WSE-3 integrates 4 trillion transistors and 900,000 AI cores across 46,225mm², delivering 21 petabytes per second of on-chip memory bandwidth ³ ⁵. Compared to NVIDIA’s B200, that translates to 19x the transistor count and 28x the compute capacity ³. On open models, Cerebras also reports inference speeds up to 15x faster than GPU-based setups ⁴.

This partnership has been building momentum for months: OpenAI signed a 750MW ultra-low-latency inference infrastructure agreement with Cerebras in January, and Cerebras raised $5.5 billion in its May Nasdaq debut — the largest IPO of the year ⁴ ⁶. Looking ahead, Cerebras maintains that the architecture scales linearly with model size, preserving its speed advantage for future generations of frontier models ¹.

What the Numbers Show, and What Remains to Be Verified

The published figures are striking. On Humanity’s Last Exam — a 2,500-question benchmark designed to test PhD-level domain expertise — GPT-5.6 Sol Ultrafast completed the entire evaluation suite in 11 hours 11 minutes, whereas Claude Fable 5 required 78 hours 27 minutes under a comparable high-reasoning setup, nearly 7x longer ¹. Cerebras also reports generation speeds 11x faster than Fable 5 and 5x faster than Opus 4.8 Fast mode based on Artificial Analysis figures ¹. On GDP-Val, a benchmark tracking economically valuable knowledge work, Ultrafast delivered a 5.6x end-to-end speedup with zero degradation in output quality ¹. However, these evaluations were run internally by Cerebras (on July 10 and July 31, respectively), and reference baselines were measured at different points in time under differing reasoning parameters ¹. Both Cerebras and OpenAI note that these performance comparisons rely on third-party benchmarks or internal testing, and real-world results may vary across workloads, system configurations, and model architectures ¹ ².

Key Questions Ahead

This release signals that the inference speed race has expanded from model optimization into silicon design. As multi-agent loops, tool invocations, and parallel sessions become standard workflows, token latency itself has emerged as the defining productivity bottleneck ¹ ². Wafer-scale computing has firmly staked out its position as a distinct alternative to conventional GPU clusters, and OpenAI’s 750MW contract reflects the scale of that conviction ⁴. The verification agenda is now clear: whether preview-stage vendor metrics hold up under independent testing, and whether capacity and pricing remain competitive against GPUs at general availability, will determine Ultrafast’s long-term impact.