OpenAI says its Jalapeño inference chip delivered 104.3 times as much DeepSeek R1 throughput per kilowatt as an Nvidia GB300 system while matching the rival's fastest per-user token rate. That number is striking, and unusually easy to misread. It describes one coordinate on a latency-throughput curve, where the comparison system had almost no capacity left at its minimum time between tokens. Across the broader peak-throughput tests, OpenAI reports a smaller 1.5 to 1.9 times advantage per watt. For developers building agents, the important result sits between those figures: Jalapeño appears able to keep responses fast as concurrent demand rises, though OpenAI has yet to publish the cost or production data needed to turn a benchmark lead into an API price claim.
The August 25 results are the first detailed measurements for OpenAI's first custom inference chip. Jalapeño ran three public models: GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. OpenAI compared it with commercially available Nvidia systems using the public InferenceX test format, then reported both latency and throughput rather than collapsing performance into a single tokens-per-second score.
That choice matters because an inference server can make one user fast by serving fewer users, or increase total output by batching more requests and making each user wait. A useful comparison has to show what happens across that trade-off. Jalapeño's first results say it moves the curve in both directions. They do not yet tell buyers what the curve costs.
What the 104.3x result measures
On DeepSeek R1, OpenAI reports 19,641 mixed tokens per second per kilowatt at peak throughput, compared with 11,781 for a GB300 setup. That is a 1.7 times lead. End-to-end latency was 1.65 seconds for Jalapeño and 5.99 seconds for the comparison system, a 3.6 times difference. Its minimum time between generated tokens was 1.43 milliseconds, versus 5.90 milliseconds, according to the company's result table.
The 104.3 times figure comes from a narrower test. OpenAI asked how much throughput each system could sustain at the GB300 result's minimum time between tokens, about 5.90 milliseconds. Jalapeño produced 12,258 mixed tokens per second per kilowatt at that response rate; the comparison managed 118. The ratio is real within the disclosed setup, but it is a comparison at the extreme edge of one system's curve. It should not be read as Jalapeño doing every DeepSeek workload 104 times faster.
The same pattern appears with the other models. At the previous system's minimum time between tokens, OpenAI reports 53.7 times more throughput per kilowatt for GPT-OSS and 56.1 times more for Kimi K2.5. At peak throughput, the leads are 1.9 times and 1.5 times respectively. Those paired measurements describe a chip that retains more serving capacity at low latency, rather than a general two-order-of-magnitude jump.
This distinction is especially relevant to coding and research agents. Their model calls often happen in sequence: inspect a file, reason, call a tool, read the result and repeat. OpenAI's benchmark post notes that delays accumulate across those steps. Higher batch throughput can lower serving cost, but it does little for an individual agent if each turn enters a slow queue. A system that holds throughput while keeping time between tokens low could support more simultaneous agent sessions without stretching every workflow.
The power denominator needs context
OpenAI normalized the results using each accelerator's published chip power rating. Jalapeño is rated at 700 watts, while the tables use 1,200 watts for GB200 and 1,400 watts for GB300. The company also says Jalapeño drew no more than 550 watts continuously during the tested workloads. Using its 700-watt rating in the calculation therefore does not give Jalapeño credit for that lower observed draw.
Package ratings still differ from the electricity consumed by a deployed rack. CPUs, memory, switches, cooling and power conversion all add to the bill. The InferenceX methodology explains that an all-in utility measurement includes those supporting systems and facility overhead. OpenAI's published Jalapeño tables are labelled with package power ratings instead. They establish a chip-level throughput-per-kilowatt comparison under the stated normalization, not a data-centre energy audit.
Price is another missing denominator. The InferenceX team treats total cost per million tokens as the purchasing metric because faster hardware can still be a worse deal when its hourly ownership cost is higher. OpenAI has disclosed no Jalapeño price, manufacturing yield, rack cost or expected service life. It plans to use the hardware inside its own infrastructure, so external customers may only encounter the economics indirectly through API availability, latency or pricing. None of those customer effects has been quantified.
The benchmark should also be read as a vendor result. InferenceX is an open test project that continually reruns public software and hardware combinations; its maintainers say software changes can move results within days. Jalapeño itself is first-party hardware and is not available for outside operators to rerun these exact tests. OpenAI published the model shapes, comparison systems, power normalization and operating-point results, which makes the claim more inspectable than an unexplained peak score. Independent reproduction will have to wait for access to the system or fuller run artefacts.
Why a purpose-built inference chip can gain ground
Language-model serving alternates between different bottlenecks. Processing the prompt, called prefill, uses substantial compute. Producing tokens during decode leans more heavily on memory bandwidth. Splitting a large model across chips adds communication delays, and the key-value cache for an active conversation has to remain available as generation continues. Idle compute during data movement can erase impressive arithmetic specifications.
OpenAI says Jalapeño was designed around those phases as one chip, memory, network and rack system. Model state can be placed explicitly and kept local, while compute and communication resources are assigned for each inference phase. The company calls the accelerator fungible because the same resources can be rearranged as a workload shifts between prefill and decode rather than dedicating separate hardware to each job. The benchmark results offer early evidence for that design, particularly where low latency and concurrency collide.
Jalapeño is aimed at inference, not model training. That limits the scope of any claim that OpenAI has replaced Nvidia. The company says it will continue deploying Nvidia accelerators and hardware from other partners for training and inference. Custom silicon gives it another serving option for high-volume workloads, while frontier training still depends on a wider supplier portfolio.
Broadcom and Celestica helped turn OpenAI's architecture into hardware and rack systems. In their June launch announcement, the companies said the design reached manufacturing tape-out in nine months. Broadcom contributed chip implementation and networking technology; Celestica worked on boards, racks and production systems. Engineering samples were then running at the target frequency and power, but the detailed performance report had not yet arrived. The August data fills part of that gap.
OpenAI also credits its models with shortening design and verification loops. For software bring-up, it says Codex with GPT-Astra helped bring GPT-OSS, DeepSeek R1 and Kimi K2.5 to high performance within two months, even though those models were outside the original production plan. On selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than existing code written by human specialists. OpenAI explicitly limits that comparison to those blocks; it is not a full-model speedup.
Deployment will be the harder test
OpenAI plans to begin installing Jalapeño in its compute infrastructure by the end of 2026. Production qualification, software work and validation across more models are still underway. A second generation is in development, and the company says it has started shaping a third. That roadmap indicates intent, while the first-generation rollout will show whether the lab results hold across uptime requirements, changing model kernels and real request traffic.
The next useful disclosures are operational: how many racks enter service, which workloads they handle, measured wall-power consumption, availability under sustained load and cost per completed request. API users should watch latency and rate limits as deployment begins, not assume the 104.3 times point will map directly to their bill. Jalapeño has produced a credible first benchmark curve. Production data will determine how much of it reaches developers.