What happened
Amazon has opened TorchNeuron, software that lets AI developers run code written in PyTorch on its own Trainium chips, to all developers. The tool had been in private beta since late last year and is now available as a public beta.
Why it matters
Nvidia dominates the AI chip market largely because much of the industry's AI software was built around its CUDA platform. Adapting that code to other hardware can take months, which has made many companies reluctant to switch to alternatives.
According to Amazon, TorchNeuron runs PyTorch code on Trainium after only a basic configuration change, without modifying the code. "If you know PyTorch, then you know Trainium," said Colin Brace, vice president of technology software at Annapurna Labs.
Similar moves across the industry
Google is working on TorchTPU, a similar effort to run PyTorch natively on its TPU chips. Microsoft said last week that it plans to run most of its AI services on its homegrown Maia chips. Amazon is also reportedly in talks to acquire Decart, a startup that makes software to move AI models built for Nvidia chips to alternatives such as Trainium, in a deal that could value it at about $7 billion.
Usage and open-source plans
Amazon plans to release TorchNeuron as open-source software on Tuesday, October 20, in conjunction with the PyTorch Conference in San Jose, California. More than 30 companies and internal Amazon teams used the tool during the private beta. Speech AI startup Deepgram said its benchmarks showed Trainium processing about 60% more audio per dollar than its existing GPU setup.
