TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Hugging Face has published a model-support Space and dataset showing which selected Transformers models it has tested on AWS Trainium2 and how to run them. A pass confirms a short test under specified conditions; it does not establish throughput, performance at longer generation lengths or support for every model on the Hub.
Hugging Face has introduced a searchable model-support Space that reports whether selected Transformers models run in its tests on AWS Trainium2, alongside the tested software versions and a script for running each model. The resource gives developers a practical compatibility check, but a passing result is limited to a short test and does not establish a model’s speed or performance in production.
The Space presents a curated selection drawn from popular Hub models, chosen using downloads and trending scores. Users can also search the Hub’s Trending or Downloads views by model name; those lists follow Hub rankings for each task and can include models that have not been tested. The display reports the test result, the number of Trainium cores used, wall-clock run time including compilation, and peak device memory. It also lets users select the Neuron compiler and Transformers versions represented in the results.
For a model with a test entry, the Space provides the software stack and run.py script used. According to Hugging Face, the recipe loads the model through Transformers’ from_pretrained interface, shards it when required, and generates with a static cache and torch.compile using the Neuron backend. The report says the example setup targets a Trainium2 instance with the relevant stack, such as an AWS Neuron Deep Learning Container. A first run compiles the model graphs; users can instead retrieve the corresponding compiled graphs from the public cache using a command supplied by the Space.
The results are also published in the aws-neuron/torch-neuronx-model-support dataset, with Parquet files organized by Neuron compiler and library versions. Each row includes the model result, any failure error, software-stack versions and the script used. This makes the information available for direct querying as well as through the Space’s interface.
A Compatibility Check, Not a Benchmark
For teams evaluating Trainium, the resource can reduce uncertainty before they spend time setting up an instance and adapting a model. A test entry can show whether a particular model and software combination completed Hugging Face’s test, what resources it used, and how to reproduce the run. Publishing the underlying dataset also gives developers a way to inspect results across compiler and library versions, rather than relying only on a static compatibility list.
The boundaries matter just as much as the pass status. Hugging Face says a test generates 32 tokens, and a PASS does not report throughput or establish behavior over longer generations. Wall time includes compilation, which can affect a first run and is distinct from steady-state inference speed. Results can help identify a starting point for evaluation, but teams still need to test their own workloads, generation lengths and performance requirements.
The distinction is relevant because a model being available on the Hub does not by itself show that it works with a particular hardware backend. The Space is a record of specific tests on a specific stack, not a guarantee that every version, configuration or deployment will work. Its strongest use is as a reproducible compatibility reference with concrete limits.
AWS Trainium2 EC2 instance machine learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How Hugging Face Tests Trainium
The announcement builds on Hugging Face’s earlier introduction of TorchNeuron, its native PyTorch backend for AWS Trainium, distributed through the torch-neuronx package. Hugging Face says Transformers uses its own model implementation with this backend, which means models may be candidates to run on Trainium; the new Space addresses the more practical question of which ones have actually passed the organization’s tests.
According to the report, each test loads a model on a Trainium2 instance and shards it across the smallest number of Trainium cores that can hold its weights. Most models use tensor parallelism. Mixture-of-experts models without a tensor-parallel plan use expert parallelism alone. Testing is limited to models that fit on four Trainium2 devices, or at most 16 cores. For a single-core run, generated tokens are compared with output from a CPU run of the same checkpoint; for sharded runs, a small judge model checks whether the text is fluent English.
At the time described in the report, coverage was restricted to Transformers models in text generation and image-text-to-text. Hugging Face said it planned to add Diffusers, sentence transformers and other Transformers tasks. The selected list and supported software versions should be read as a snapshot, not as evidence that the same coverage applies to every current model or stack.
“A PASS says nothing about throughput, or about longer generations.”
— Hugging Face, in its TorchNeuron model-support report
AWS Neuron Deep Learning Container Trainium2
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Coverage and Performance Limits
The report does not establish that every model listed on the Hub has been tested, or that a model without a result is incompatible. Some models are absent because they were not prioritized, while the Space may identify incompatibility in other cases. The curated selection, rankings and testing scope are limited, and the report does not specify a schedule for broader coverage.
Nor do the results answer how quickly a model will generate tokens under a reader’s workload. The stated test checks 32 generated tokens; it does not measure throughput, longer generations or production behavior. Results are tied to the listed Neuron compiler and library versions, and may not transfer unchanged to other versions or configurations. Hugging Face’s report does not provide a general performance comparison with other hardware.
The validation also differs according to how a model is run: single-core output is compared with a CPU run of the same checkpoint, while sharded output is reviewed by a small judge model for fluent English. Readers should treat these methods as the report’s stated checks, not as evidence of a full evaluation of output quality or model accuracy.
AWS Trainium2 developer cloud compute credits
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Broader Tests and Local Runs
Hugging Face said it plans to expand the tested coverage to Diffusers, sentence transformers and additional Transformers tasks, though it did not give a timetable in the report. Until then, users can search the available views, check the model’s result and inspect the recorded software stack before attempting a run.
For a reproducible check, the report directs users to install the packages listed for a model on a compatible Trainium2 setup and run its supplied script. The first run compiles model graphs; the Space also provides a command to fetch matching graphs from the public cache for users who want to skip that step. Developers can query the public dataset for version-specific records and failure details. Any deployment decision still requires testing with the intended software versions, workload and performance targets.
Key Questions
What does the Trainium model-support Space show?
It lists test results for selected Transformers models on AWS Trainium2, including core count, wall time with compilation, peak device memory and the script and software stack used.
Does a PASS mean a model is fast on Trainium?
No. Hugging Face says the test generates 32 tokens and that a PASS does not indicate throughput or performance on longer generations. The recorded wall time includes compilation.
Does a missing model result mean the model is incompatible?
Not necessarily. The report says some models were not prioritized for testing, while others may be left out because they are incompatible. The Space may explain why a model was omitted.
Which models and tasks were covered in the report?
At the time of the report, the tests covered selected Transformers models for text generation and image-text-to-text, limited to models fitting on four Trainium2 devices. Hugging Face said it planned broader task coverage.
Can developers run a tested model themselves?
Yes. The Space provides a run.py script, required packages and the tested Neuron stack for each tested model. A first run compiles the graphs; users can also retrieve matching compiled graphs from the public cache.
Source: rss
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
