CUDA's Castle Without Its King

Jensen Huang resigned today, and NVIDIA's CFO, Colette Kress, took over as interim CEO. Two days earlier, Anthropic opened its IPO roadshow with a presentation that made me think differently about the business she inherits. One of NVIDIA's customers had used its own AI to do much of the work that makes switching away from NVIDIA so expensive. NVIDIA's announcement · Anthropic's roadshow

CUDA (Compute Unified Device Architecture) is NVIDIA's programming platform for running calculations in parallel on its GPUs. AI teams tune small programs, called kernels, for those chips. Switching to AMD can mean rewriting and testing that code before anyone knows whether the cheaper chip saves money. NVIDIA's guide to CUDA

Anthropic says Claude Mythos Preview wrote or adapted almost all the kernels for an experiment on AMD's Instinct MI355X. In about three days, one Claude model was processing prompts at a speed Anthropic considered useful. The company estimated six to twelve months for the traditional approach. The small print matters here. Prompt processing is the part where Claude reads your question. How fast it writes the answer, what happens when thousands of people arrive at once, and what the whole service costs to run are still open questions. Anthropic's presentation, about 20:42

I keep coming back to what those three days buy an engineering team. Ask for six months to try a different GPU and someone quite reasonably asks which product you plan to delay. Ask for three days and you might have a result before the next planning meeting. Even a mixed result is useful. If AMD turns out to be good at reading long documents, the team can start there and keep the rest of its work on NVIDIA.

There's a detail in the July agreement between AMD and Anthropic that I paid less attention to at the time. The headline was up to two gigawatts of MI450-series GPUs. Further down, the companies said they would use Claude to improve ROCm, AMD's software platform. If a fix makes its way into ROCm, the next team to try AMD can use it too. That's part of what made CUDA so attractive in the first place. You got to start where another engineer had left off. AMD's partnership announcement

Here's what that could mean for NVIDIA's next sale. Suppose a batch of documents costs Anthropic $100 to process on NVIDIA and $80 on AMD, including the extra engineering time. Those are example numbers. With both machines tested, Anthropic can ask NVIDIA for a better price. If NVIDIA holds firm, Anthropic can send the next batch to AMD. If NVIDIA offers a discount, it can keep the work.

Working in robotics, I'd still pay extra for equipment my team knows how to fix on a Friday night. CUDA has earned a premium. What Huang's successor has to find out is how much of that premium customers will keep paying once they can put a number on the alternative. NVIDIA could sell Anthropic every GPU it planned to sell and earn less on each one. That's where I'd look for the first sign that CUDA's hold is weakening.