Google Cloud launches Agent Compute Engine and open-sources llm-d 1.0 and TorchTPU

Google Cloud describes Agent Compute Engine as a compute service for autonomous agents, providing hardware-isolated microVM sandboxes alongside Agent Gateway and Agent Registry for governance. The company cites sub-second activation, copy-on-write memory sharing, enterprise identity and security controls, and cross-cloud observability as its main benefits.

Google also open-sourced llm-d 1.0, a Kubernetes-native serving stack that groups model servers into one coordinated pool across mixed Google Cloud TPU and GPU fleets. It adds cache-aware routing, tiered KV cache offloading and priority-based flow control, which Google says improve context reuse and reduce time-to-first-token. The project was co-founded by Google and partners.

Working with Meta, Google open-sourced TorchTPU, a native PyTorch stack that brings TPUs into PyTorch alongside GPUs and CPUs. According to the update, developers can train, serve and deploy models on TPUs using Hugging Face, vLLM and TorchTitan.

Teams running agents can use microVM sandboxes with fast startup instead of building their own isolation layer. Check the availability status of Agent Compute Engine in Google's documentation before planning around it.

If you serve models across TPUs and GPUs, llm-d's routing and KV cache features offer a ready-made serving layer to evaluate. TorchTPU means PyTorch workloads can target TPUs without moving to a different framework.