Amazon opens TorchNeuron to run PyTorch code on Trainium

On 20 October, at the PyTorch Conference in San Jose, AWS opened TorchNeuron to all developers as a public beta. It is an open-source backend that lets PyTorch code run on Amazon's Trainium chips, released under Apache 2.0 with support for Trainium2 and Trainium3. It had been in private beta since late last year; one report says over 30 early customers use it.

Existing PyTorch code can target Trainium with a one-line device change. Amazon's demo ran the same training script on a standard chip and on Trainium, and said the model learned at the same rate on both. Eager mode, torch.compile, FSDP2, tensor and pipeline parallelism and FlashAttention are reportedly supported natively.

Some PyTorch features such as dynamic input shapes are not yet supported, and about 22,000 PyTorch tests are skipped, per the PyTorch blog. One report says 89% of models run. The announcement is dated 19 or 20 October depending on the source.

Nvidia's real moat is CUDA: code written for it is costly to move. Every cloud is building its own chip, and their success depends on software. TorchNeuron aims to cut switching costs toward zero, as DeepSeek's DeepInfer does for Huawei chips the same week.

Teams training or fine-tuning models on AWS are most affected. Most rent Nvidia GPUs today; Trainium may be cheaper, but porting time and risk held them back. If TorchNeuron removes that barrier, AWS customers could get more compute for the same budget, while Nvidia sees one of its largest customers accelerate a shift to its own silicon.

• Keep code portable before adding chip-specific optimisations.
• Compare costs by running the same workload on different chips.
• Check unsupported features before moving production work to beta software.