OpenAI Jalapeño: Custom Inference ASIC Outperforms Nvidia Blackwell in Power Efficiency

ArticleBryan Shan, Myron Xie, Jordan NanosNews

At the Hot Chips conference, OpenAI revealed details of its custom LLM inference chip codenamed Jalapeño. Developed in partnership with Broadcom, the ASIC outperforms Nvidia Blackwell and Vera Rubin in token throughput per megawatt according to initial lab benchmarks by SemiAnalysis.

Key points

  1. OpenAI designed Jalapeño from a blank slate with Broadcom, moving from initial hiring to manufacturing tape-out in approximately 16 months.
  2. Rather than being locked to proprietary OpenAI architectures, Jalapeño is a general-purpose inference ASIC capable of running models such as DeepSeek R1, Kimi-K2.5, and GPT-OSS.
  3. Built on TSMC N3P with HBM4, the chip achieves record-setting throughput per megawatt using Single Token Prediction (STP) without relying on Multi-Token Prediction or speculative decoding.
  4. Against Nvidia's Vera Rubin, Jalapeño leads in energy efficiency (tokens per joule) and matches it on total cost of ownership (TCO).
  5. Caveats apply: verified tests relied on OpenAI-provided lab runs using 8k1k workloads (InferenceX), without comprehensive multi-turn AgentX benchmarks, and the chip currently exists only as engineering samples.

Why it matters

Datacenter power capacity, rather than hardware budget or space, has become the primary constraint for scaling AI inference. A chip that maximises tokens per megawatt directly improves unit economics and reduces reliance on Nvidia's pricing power.

Analysis

Jalapeño demonstrates that aggressive hardware-software co-design accelerated by AI can yield competitive first-generation silicon. OpenAI avoided over-specialising on separate prefill and decode hardware, opting for a homogeneous pool adaptable to shifting prompt-to-completion ratios. While lab benchmarks on A0 silicon and B0 specs (700W TDP, 13.4 MXFP4 PFLOPs) are impressive, real-world validation against production-scale agentic caching loads (AgentX) remains the key hurdle before volume deployment.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Link:Etched

    Etched Unveils Frontier Inference Clusters and Secures $800M in Funding

    AI hardware startup Etched has introduced dedicated inference clusters designed specifically for frontier models. Following the delivery of A0 silicon from TSMC N4P, the company announced $800M in raised funding and $1B in customer demand.

    AI & LLMs· Announcement

  • Article:Julie Bort

    Hardware startup Etched doubles valuation to $21bn in a month

    AI hardware startup Etched has raised $700 million at a $21 billion valuation. The round was led by quantitative trading firm Jane Street following tests and purchases of the startup's inference hardware.

    AI & LLMs· News

  • Repository:ashhart/TensorFold

    TensorFold: LLM Inference Engine for Apple Silicon and Modern NVIDIA GPUs

    TensorFold is an open-source inference engine designed to serve large language models on Apple Silicon (via MLX/Metal) and newer NVIDIA GPUs (Compute Capability 8.9+). It provides OpenAI- and Anthropic-compatible endpoints with model-tailored kernels and speculative draft verification.

    1.1kstarsPython

    AI & LLMs· Tool

  • Article:Nate B. Jones

    AI News & Strategy Daily: Nate B. Jones on Agent Economics, Hardware, and Model Tactics

    In his podcast AI News & Strategy Daily, strategist Nate B. Jones assesses key AI industry shifts from August and September 2026. Topics include effort configurations in Claude Fable 5.1, switching costs tied to custom hardware like OpenAI's Jalapeño chip, Apple's M6 local-AI strategy, and the operational overhead of managing autonomous agents.

    AI & LLMs· Opinion

  • Link:Modular

    Modular: Unified AI Inference Stack from Kernel to Cloud

    Modular presents its AI inference platform designed to execute workloads portably across hardware architectures including NVIDIA, AMD, TPUs, Trainium, and Apple silicon, combining the MAX serving framework with the Mojo language.

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.