Source: developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale
Agent Substrate snapshots an idle agent's sandbox, including memory and filesystem state, frees its worker, and restores the agent onto whichever worker is free when the next request arrives. The runtime targets coding assistants, reinforcement learning post-training with bursty CPU-heavy environments, and short-lived GPU sandboxes for research. Kubernetes cold starts are the problem it addresses, since creating a Pod per agent step is too slow for sandboxes that run short commands hundreds of times per second.
Upstream work described in the post includes request parking in the Agent Substrate router, which queues a request when all workers are busy instead of returning an error. Other contributions cover per-container CPU and memory limits for microVM sandboxes and a vendor-neutral design for pausing and restoring GPU state using Kubernetes Dynamic Resource Allocation. NVIDIA GPUs are named as the first target for that device support.
On the inference side, NVIDIA Dynamo's router uses session identity and KV cache location to keep related model calls on workers that hold reusable context. Dynamo sends signals such as projected time to first token and GPU-pool saturation back to Agent Substrate, which uses them to decide when to suspend or resume an agent. The post says Agent Substrate is still evolving, its APIs are changing, and parts of the Dynamo integration are experimental.
Why it matters if you build agents
Developers running many agent sessions on Kubernetes can look at checkpoint and restore of idle sandboxes as an alternative to keeping one warm Pod per agent or paying a cold start on every step.
The post says the GPU state restore design can only restore onto matching hardware and may fall back to a cold start, so teams planning GPU sandboxes should not assume suspended state will always resume.
Because the APIs are still changing and parts of the Dynamo integration are experimental, teams adopting it should expect breaking changes.
Get stories like this every weekday
One short email with the day's agent tooling news.