IBM brings its Spyre accelerator into PyTorch and vLLM workflows

torch-spyre is a PyTorch backend that compiles ordinary PyTorch code into an executable plan for Spyre. When a model contains an operation the stack cannot yet lower effectively, an adapter swaps in a mathematically equivalent form, so the model's computation stays the same. IBM says moving to TorchInductor cut the custom compiler code its team maintains, with the resulting compiler about seven times smaller than the system it replaced.

Spyre is an out-of-tree accelerator, so it relies on vLLM extension points such as CustomOp and PluggableLayer. The vLLM community is building hardware-agnostic layers, with early support landed and the Spyre pathway validated on Gemma 4, Qwen3 and Granite 4.2. On NVIDIA H100 GPUs, those layers delivered total token throughput within 3.4% of the native implementation.

IBM says the goal for IBM Z, LinuxONE and Power11 users is to take models from Hugging Face and use largely the same PyTorch workflows they would use on Nvidia GPUs. Spyre's PyTorch integration also lets it plug into llm-d, which runs distributed inference on top of vLLM and supports multiple accelerator types.

Developers targeting IBM Z, LinuxONE or Power hardware can keep standard PyTorch and vLLM code paths instead of relying on a small catalog of specially packaged model weights. Teams planning agent workloads on Spyre should check which models and accelerator configurations have been exercised, since IBM is adding end-to-end model tests against CPU or GPU references. Hardware-agnostic vLLM layers may also reduce the rework needed when moving inference between accelerator types.