# OpenAI's Chili Pepper Chip: When Complexity Can Be Outsourced to AI, the Rules of Chip Design Change > Notes after listening to TechWave EP152. Starting from Jalapeño beating Blackwell on a public inference benchmark, three layers: how far those numbers can actually be pushed, why a chip that switches its own operating modes saves power, and the key shift — once programming complexity can be handed to AI, all the trade-offs chip designers avoided because they were 'too painful' are back on the table. Educational content, not investment advice; no stock recommendations or price targets. Published: 2026-09-01 Locale: en Tags: semiconductors, AI chips, OpenAI, industry trends, TechWave ![A long aisle deep inside a data center, indicator lights on racks receding toward a distant vanishing point, and in the foreground a gloved pair of hands holding a single chip up under an inspection lamp, its edge catching a faint red glow](/covers/techwave-2026-08-31-ep152-openai-jalape-o-cover.png) > The wondrous, the strange, the extraordinary sights of this world lie always in remote and difficult places, where few men ever arrive; and so without resolve, no one reaches them. > > —— Wang Anshi, "Record of a Journey to Baochan Mountain" (Northern Song, 1054; translation mine) Remote places stay empty not because they are unrewarding, but because the road is hard. And the moment someone paves that road, the place that was empty because of the difficulty gets crowded very fast. That, in essence, is what this episode describes happening to chip design. ## What the Episode Covers TechWave's August 31, 2026 episode (EP152) walks through the freshly published benchmark results for Jalapeño, OpenAI's in-house inference chip. The chip is named after the pepper, and the host cheerfully admits he orders extra jalapeños on every Subway sandwich — which sets the tone nicely for what is basically a long-awaited "we finally have numbers" moment. The interesting part isn't the win itself. It's three layers underneath: how much weight those numbers can actually bear, what the hardware does that nobody used to do, and why it's suddenly feasible. ## Key Points **1. A year-old argument finally has public numbers.** For roughly a year people have speculated that purpose-built AI chips would beat NVIDIA's general-purpose GPUs on energy efficiency. But the two credible contenders — Google's TPU and Amazon's Trainium — never submitted public benchmark results. Jensen Huang spent the year hammering exactly this: the tests are sitting right there, we ran all of them, our scores are online, so why won't you run them? Nobody answered, and the industry roughly concluded they'd run them privately and not liked what they saw. Jalapeño is the first to actually submit. **2. The scores are strong.** The benchmark is SemiAnalysis's InferenceX, currently the most credible inference test available. At equal power draw, throughput leads by 1.5–1.9x. End-to-end latency — from a user's request arriving to the full answer completing — is 1.7–3.6x faster. If you optimize purely for latency regardless of cost, the ceiling is 2.1–4.1x. And it didn't only run OpenAI's own model: three models across small, medium and large sizes with differing architectures, which is unusually disciplined for a first-party benchmark submission. **3. The host's four caveats matter more than the multiples.** Power is computed from the chip's rated package TDP, not measured at the wall. The test scope is a single rack, not a full AI factory — and full-factory co-design is precisely where NVIDIA is strongest. The workload is a fixed long-input, short-output shape, while the dominant real-world pattern today is agentic back-and-forth. And the comparison is against Blackwell, when by process node and memory generation the fair opponent is Rubin. His read: these narrow the gap, but don't overturn the conclusion. **4. The prettiest hardware idea is one chip switching its own mode.** Inference splits into phases with completely different hardware appetites. Processing your whole prompt is compute-hungry — every token's relationship to every other token gets computed at once. Emitting the answer token by token is bandwidth-hungry — each token requires hauling model weights out of memory again. The conventional answer is different chips for different phases, which forces intermediate state to be shipped between chips, and leaves capacity idle whenever the phase mix shifts with user behavior. Jalapeño lets one chip reconfigure itself between modes: nothing gets shipped, and units not needed in the current mode are simply switched off and draw no power. The saving is transport cost plus idle cost. **5. The real dividing line is the spatial architecture.** On a normal GPU, work assignment is done by hardware — an on-chip scheduler hands jobs to whichever unit is free. That makes software easy to write, portable across cards, and gives a floor on utilization. The cost is that the scheduler only sees the present moment. The host's example: if core one's result will later be combined with core three's, the scheduler doesn't know, so both write results back to memory and a third core is dispatched to fetch and merge them — a long detour. A spatial architecture lets software specify exactly where each piece of data lives, which core computes it, and who it goes to next. Much higher ceiling, because every path can be the shortest one. The price is that software complexity explodes, and badly written software is *worse* than the GPU: give one core a long job everyone else is waiting on, and the whole batch stalls. **6. Nobody designed chips this way before because nobody could write the software.** The one outside firm with real exposure describes OpenAI's low-level code as being like writing assembly, with single kernels running past three thousand lines. Where do you hire engineers to write that? You don't — which is why the industry never traded that much programming pain for a bit more performance. OpenAI could take the trade because the complexity is now borne by AI: humans write the high-level function correctly, and the data-flow optimization and shortcut routing are left to the compiler and an AI optimizer. The evidence is how quickly they produced custom kernels for three architecturally different models. **7. The most counter-intuitive line in the whole story.** Historically, squeezing out performance meant extreme specialization — hard-wiring the chip to one model architecture and giving up generality. Jalapeño didn't do that. It runs other companies' models, and OpenAI even demoed it running DOOM to prove the architecture isn't frozen. It got both relative generality *and* optimized data flow, because the trade-off between them — "the software is too hard to write" — was eaten by AI. As a footnote: nine months from architecture lock to tape-out, on a first-generation chip. ## Going Deeper ### "The news says someone beat NVIDIA. Should I be doing something about my position?" That's the first thought, and it's almost always too fast, because "beat" is an extremely compressed word in chip comparisons. Unpack it and there are at least three questions about scope. **First: beat on which axis.** The win is throughput-per-watt and end-to-end latency. Not raw compute, and not training — Jalapeño physically cannot train models. That matters: it entered a race NVIDIA also runs, but which represents only part of NVIDIA's business. A pure inference entrant winning the inference event is not the same thing as replacement. **Second: is the opponent the same generation.** It ran against Blackwell, but the same-process, same-memory-generation opponent is Rubin. New-versus-old comparisons are everywhere in industry news, and usually not deliberate cheating — the newer part simply has no published numbers yet. Your job isn't to cry foul, it's to apply the discount yourself. **Third: where are the test boundaries.** Single rack, not full factory. Rated power, not measured power. A fixed long-in/short-out workload, not today's dominant agentic pattern. And note that every one of those boundaries happens to disadvantage NVIDIA, because full-factory co-design — its strongest asset — simply cannot express itself inside a single-rack test. After those three questions the conclusion isn't "the news is fake." It's "the news is real, and it supports a smaller claim than the headline." That's what reading skill buys you: not rejecting good news, but knowing where its edges are. And the genuinely alarming detail here isn't any of the multiples — it's the two phrases "first generation" and "nine months." ### "So is the moat gone or not? The same episode seems to say both." That confusion comes from treating "moat" as one object. It's several layers of different material, and the layer being dissolved isn't the layer still standing. **Software-difficulty moats:** something is hard to replace because there's too much code to rewrite, the ecosystem is too thick, the learning curve too steep. This layer is eroding, for exactly the reason this episode describes — once programming complexity can be handed to AI, "hard to write" stops being a wall. The host pushes it all the way: anything whose value came from development difficulty will gradually stop having that value. **Physical and interconnect moats:** chip-to-chip links, rack-to-rack interconnect, cooling. Years of physical engineering, not replicable by writing software. The host names this as NVIDIA's real moat. **Capital and installed-base moats:** deployed scale creates its own inertia — toolchains, operational experience, partners all stuck to it. Slowest layer to move. With that split, the episode stops contradicting itself. Jalapeño is attacking layer one and barely touching two and three. The fact that it wins convincingly in a single rack but might win by far less at full-factory scale is itself evidence that layer two is intact. This decomposition generalizes. For any competitive advantage you're assessing, ask: **what material is this moat made of — "it's a hassle for others," or "others physically cannot"?** The shelf life of the first kind depends entirely on how fast that hassle gets eaten by tooling. ### "The host is revising his own timeline. Should I revise mine?" The closing stretch is unusually honest. He says his prior view was that NVIDIA was safe for three years — carried by interconnect, full-factory co-design, and capital plus installed base — with real uncertainty only after that. Having seen these results, he says he may need to revisit that timeline. Notice precisely what he revised. Not "I was wrong, NVIDIA is finished." Not "this changes nothing." He revised the **timeline**. The structure of his view — three moat layers, the software layer being the fragile one — is unchanged. What changed is that the fragile layer is eroding faster than he assumed. That distinction is useful for your own thinking, because new information only has three possible destinations: change the conclusion, change the timeline, or change the invalidation conditions. Most people only use the first and the last — either overturn everything or pretend they didn't see it. In practice the middle one is the most common: the framework was right, the speed estimate wasn't. And when the speed estimate is wrong, what usually needs adjusting isn't whether you hold something — it's **your checkpoints and invalidation conditions**. If your plan was "revisit in three years," move the checkpoint forward and write down specifically what you're watching: second-generation results, whether the next comparison uses a same-generation opponent, whether beyond-single-rack testing appears, whether the other players start submitting scores. Those are observable and checkable, unlike a vague sense that something shifted. Worth noting: the other two purpose-built chips still submitted nothing. If they continue not to, now that someone has gone first, that silence is itself information — not about Jalapeño, but about them. ## Further Reading - TechWave EP152 (August 31, 2026): the source for this piece, host Harry's full breakdown of Jalapeño - OpenAI's official blog: the complete benchmark data, which the host describes as extremely detailed and complex — go to the primary source if you want the numbers - Hot Chips: the annual chip and hardware conference where OpenAI disclosed additional technical detail - SemiAnalysis's InferenceX and AgentX: currently among the more credible inference benchmarks; the latter targets agentic workloads and was not run this time ## The One Thing to Take Away **When the difficulty of something gets outsourced, whatever value rested on that difficulty disappears with it.** The real message of this episode isn't whose chip is faster. It's that a price nobody was willing to pay — three-thousand-line low-level kernels, humans hand-routing every piece of data — is now being paid by something else. Once a price is absorbed, every option that was previously "too painful to bother with" comes back onto the table at once. And simultaneously, every advantage built on "this is hard, so nobody can catch me" starts loosening underfoot. This isn't confined to silicon. It holds anywhere your value comes from something being a hassle — knowing a complicated tool, remembering a tedious process, having the patience to do the tidying nobody else wants to do. **Something you can do today:** take a piece of paper and write down one thing you currently hold that nobody competes with you for *because it's a hassle*. A work skill. Something at home only you handle. A fixed role you play in some group. Then ask yourself one question — If a tool arrived tomorrow that made this effortless, what would I have left? Whatever you can answer is the part that's actually yours: judgment, relationships, taste, understanding of what other people are going through. Whatever you can't answer is where you should start adding. Nothing needs to change immediately — knowing which side you're standing on already puts you ahead of most people.