🔍 Read the full analysis: MPS In PyTorch 2.15: Key Details To Know on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
PyTorch 2.15 adds FP8 tensor support and new performance paths for its Metal Performance Shaders backend on Apple silicon. Hugging Face’s PyTorch team reports faster decoding in two tested language models and lower peak memory in a Wan2.2 video-decoding test, but results are setup-specific. The release requires macOS 15 or later and Python 3.11 or later.
PyTorch 2.15 adds FP8 tensor support and new performance paths to its Metal Performance Shaders (MPS) backend for Apple silicon, in an update aimed at improving local model inference and other GPU workloads on Macs. The PyTorch team at Hugging Face reports faster decoding for two tested language models and lower peak memory in a video-decoding test, while cautioning that results depend on the model and setup.
In tests on an M3 Ultra using torch.compile, the Hugging Face team measured Qwen3.6-35B-A3B decoding at 61.3 tokens per second in PyTorch 2.15, compared with 9.4 in 2.14. For LFM2.5-8B-A1B, it reported 123.6 tokens per second, up from 23.6. These are results for the named models and configuration; they do not establish expected performance on other Apple silicon Macs or workloads.
The release routes eligible single-row linear layers through specialized matrix-vector kernels and fuses some bias additions, targeting the repeated operations involved in language-model decoding. Across 96 BF16 layer shapes drawn from real models, the team reports a 1.23× geometric-mean speedup. It also reports vocabulary softmax running about eight times faster in its measurements. A native grouped matrix multiplication path for mixture-of-experts models is intended to reduce GPU-to-CPU synchronization and speed up top-k routing.
PyTorch 2.15 supports torch.float8_e4m3fn tensors on MPS. The one-byte values take half the storage of BF16, and scaled matrix multiplication can use tensorwise or rowwise scaling. In another test, the team says Wan2.2 tiled VAE decoding peaked at about 8 GiB in 2.15, compared with 270 GiB in its earlier test. It also reports that an image-padding change improved a Qwen-Image-2.1 encode-decode round trip from 10.06 dB to 29.95 dB PSNR. Those figures describe the team’s tests, not universal outcomes.
What Changes for Mac Model Runs
The update targets two practical limits for running models locally: decoding speed and memory use. Faster per-token processing can make interactive language-model responses feel less delayed, while lower memory demand could allow some workflows to run with less pressure on a Mac’s available unified memory. FP8’s smaller storage footprint may also help where supported operations and scaling choices make it suitable.
The reported Wan2.2 result could matter to video-generation users because peak memory can determine whether a pipeline runs at all. But the size of that reported reduction should not be assumed for other resolutions, settings or video pipelines. Likewise, the image round-trip result is a useful quality check for one named model and metric; it does not establish image fidelity across other models or tasks. Users should compare speed, memory and output quality in their own environment before drawing conclusions about the upgrade.
Apple Silicon compatible GPU acceleration software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
MPS Work Moves Toward Metal
MPS is PyTorch’s backend for using Apple’s Metal GPU framework. Hugging Face’s PyTorch team says that some model workflows on MPS had encountered unsupported operations, performance drops or unexpectedly high memory use. Its approach has included testing models from the Hugging Face Hub and working with PyTorch maintainers and community contributors to address gaps, including replacing some MPSGraph paths with dedicated Metal kernels.
PyTorch 2.15 builds on changes in 2.14, which had introduced matrix-vector kernels for torch.mm. The new release connects eligible F.linear calls to those kernels, focusing on single-row inputs common during language-model decoding. The release also includes work on complex LU and triangular solves, QR decomposition, convolutions, padding, dropout, interpolation backward and reductions. The team has previewed further MPS work for 2.16, but those future items are not features confirmed as released in 2.15.
The Hugging Face PyTorch team described its broader aim this way: “The goal is to make your Mac a first-class place to run and train PyTorch models.” The statement describes the team’s objective; the benchmark results and compatibility limits remain the more concrete measures of what this release changes.
“The goal is to make your Mac a first-class place to run and train PyTorch models.”
— PyTorch team at Hugging Face
PyTorch compatible MacBook accessories
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How Broad Are the Benchmarks?
The published material does not provide enough detail to independently reproduce every benchmark, including full software versions, run counts and measurement conditions. The language-model figures identify an M3 Ultra and torch.compile, but do not show how performance changes across other Apple chips, models, software stacks or generation settings.
The Wan2.2 memory comparison is also tied to the team’s test; the material does not establish how it applies across resolutions or video pipelines. The Qwen-Image-2.1 result uses PSNR for an encode-decode round trip, but no broader evaluation across images or workflows is provided. Although FP8 tensor and scaled matrix multiplication support are part of the release, practical accuracy and memory effects depend on how an application uses quantization and scaling. These limits mean the reported numbers are not guarantees for every workload.
As an affiliate, we earn on qualifying purchases.
Check Requirements and Test Workloads
Users planning to run MPS need an Apple silicon Mac with macOS 15 or later; PyTorch 2.15 requires Python 3.11 or later. The release material says MPS availability checks now explain when an older operating system or a missing Metal GPU prevents MPS from being used. Developers should check PyTorch 2.15 compatibility information before changing their environments.
The Hugging Face team has previewed work for PyTorch 2.16 that includes moving attention more fully onto Metal, additional migrations from MPSGraph and further decoding improvements. These are plans, not confirmed 2.15 capabilities. For now, the practical next step is to test the release with the specific models and Mac configurations a user relies on, recording latency, memory use and output quality under comparable settings.
AI model inference optimization tools for Mac
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is new for MPS in PyTorch 2.15?
PyTorch 2.15 adds FP8 tensor support for MPS, along with performance paths for operations including eligible single-row linear layers and mixture-of-experts decoding. The release also includes changes to other operations such as convolutions, reductions and QR decomposition.
How much faster is language-model decoding?
In tests on an M3 Ultra with torch.compile, the Hugging Face team reported 61.3 tokens per second for Qwen3.6-35B-A3B in 2.15 versus 9.4 in 2.14, and 123.6 versus 23.6 for LFM2.5-8B-A1B. These results apply to the named models and test setup, not every Mac or model.
What are the PyTorch 2.15 MPS requirements?
The stated requirements are an Apple silicon Mac running macOS 15 or later and Python 3.11 or later for PyTorch 2.15.
Does FP8 guarantee lower memory use in every application?
No. FP8 values use half the storage of BF16, but the actual memory and accuracy effects depend on the operations an application uses and its quantization and scaling choices. The reported results should be tested against each workload.
What is planned for MPS after version 2.15?
The Hugging Face team has previewed possible PyTorch 2.16 work, including more attention operations running on Metal, further MPSGraph-to-Metal migrations and faster decoding. These are future plans, not features confirmed in 2.15.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
