TL;DR: OpenAI has published the first benchmark results for Jalapeño — its first custom inference ASIC, built with Broadcom. The result: 1.5x to 1.9x more AI work per watt, and 1.7x to 3.6x lower end-to-end latency compared to commercial baseline systems, tested on three open models via a public benchmark. The biggest impact is seen in interactive workloads, showing up to 4.1x higher performance. The production ramp will take place throughout 2027.
What Jalapeño Is
- OpenAI's first custom inference chip, developed in partnership with Broadcom and optimized for serving language models.
- Benchmarked on InferenceX, the public benchmark by SemiAnalysis — meaning this isn't just an internal slide deck.
- Tested models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. This demonstrates that the architecture works well with open models, not just proprietary ones.
Performance Metrics at a Glance
| Metric | Jalapeño vs. Baseline System |
|---|---|
| Work per watt (peak throughput) | 1.5–1.9× more |
| End-to-end latency | 1.7–3.6× lower |
| Interactive workloads | 2.1–4.1× higher performance |
| Kimi K2.5 1T (largest tested model) | ~1.5× performance/watt, 3.4× lower latency |
| Nominal power / measured sustained power | 700 W / ≤ 550 W |
models: [GPT-OSS 120B, DeepSeek R1 670B, Kimi K2.5 1T]
benchmark: InferenceX (SemiAnalysis, public)
power: nominal 700 W, measured <= 550 W
production: Ramp throughout 2027, majority of output Q4 2027
Why the Chip Is Fast: Co-Design Instead of Component Stacking
Jalapeño minimizes data movement — the primary bottleneck in LLM inference. The system utilizes several levers to achieve this:
- Prefill (prompt processing) is compute-intensive, while decode (token-by-token generation) is limited by memory bandwidth. Depending on the inference phase, the chip activates the appropriate combination of compute, memory, and networking.
- The KV cache is deliberately placed and kept locally, rather than constantly shuffling between resources.
- Networking is built into the architecture: large domains keep an entire workload within a connected system so that wait times between chips don't dominate latency.
Benchmarking is based on a matched user experience: an equal amount of useful work per watt at the same latency. This is the exact metric needed for agentic workloads, where sequential sub-steps compound — if one step is delayed, the total time of the task increases.
In internal tests, the performance lead increased even further on OpenAI's frontier models, suggesting that the larger and more demanding the workload, the more advantageous the architecture becomes.
What This Means for Builders
- Downward pressure on API prices. More work per watt at the same latency lowers unit costs per million tokens — the trend of recent releases (sub-dollar pricing for open models) is likely to continue.
- The biggest leap is in interactive workloads. Developers building agents with many small, sequential steps will benefit disproportionately (up to 4.1×) — making longer reasoning chains practically viable.
- Local stacks remain valid. Jalapeño is data center hardware with a 700 W nominal power rating. For data sovereignty and deterministic costs, MLX/vLLM benchmarks remain relevant; the tested open models run in both environments.
- Plan your timelines. The majority of capacity will arrive in Q4 2027 — consolidation efforts for 2027/28 can be timed around this.
FAQ
How do I know if Jalapeño is relevant to my project? Look at two numbers: work per watt and end-to-end latency at your typical prompt length. If your workload is interactive (short responses, many steps), the chip provides the biggest leverage according to the benchmarks. For long document prompts with short responses, the difference is smaller because prefill and memory bandwidth dominate latency in those cases.
Does better performance per watt automatically mean lower API prices? Not automatically, but the trajectory is clear: unit costs per token drop as efficiency rises. OpenAI has signaled it will pass these gains on to customers, and competition among open-weight models adds further price pressure. However, the only binding numbers remain those in the official API pricing table.
Will my current models run unmodified on Jalapeño? Yes, the chip serves models, not new formats. The benchmarked examples (GPT-OSS 120B, DeepSeek R1, and Kimi K2.5) are exactly the open models that local and hosted pipelines are already using today. The API syntax stays the same; only the latency and pricing curves shift in favor of the users.
Should I move away from local models because of Jalapeño? Not yet — production doesn't start until 2027, and availability will be tight at first. Check the measured latency after its first month in production against your own local benchmarks. If data sovereignty is a priority (on-prem, no external requests), local stacks will likely remain the right choice.
How does Jalapeño compare to NVIDIA Blackwell? The baseline systems in InferenceX are commercially available Blackwell generations. Within the tested operating range, Jalapeño holds the Pareto front: an equally good or better combination of throughput, efficiency, and latency. This doesn't mean "always faster," but rather: under identical latency budgets, the chip delivers more per watt — which is a more practical metric for agents.
Sources
- OpenAI — Jalapeño’s first results: https://openai.com/index/jalapeno-first-results/
- SemiAnalysis (Publisher of InferenceX): https://semianalysis.com/
- Hacker News Discussion (298 points): https://news.ycombinator.com/