# Google's Tunix adds zero train-inference mismatch for stable RL on TPU

published: 2026-11-16T16:05:47.205Z
topic: infrastructure
source: https://developers.googleblog.com/unlocking-stable-on-policy-rl-on-tpu-with-bitwise-zero-traininference-mismatch/
url: https://aiagentnews.io/stories/google-s-tunix-adds-zero-train-inference-mismatch-for-stable-rl-on-tpu

Google Developers Blog describes a Zero-TIM implementation in Tunix, its open-source JAX library for LLM reinforcement learning, that makes sampling and training produce bitwise-identical logprobs on TPUs. The approach was verified across 64 TPU v5p chips in a DP8×TP8 mesh.

Reinforcement learning for large language models is sensitive to numerical differences, because sampling and training often run through separate execution paths. Under identical weights, those paths can assign different logprobs to the same token, since floating-point rounding depends on the order of operations. The effect is easier to see in bfloat16.

The Zero-TIM implementation in Tunix aims for bitwise-identical logprobs across sampler decode, serving prefill and trainer forward. The recipe uses batch-invariant Pallas kernels within each tensor-parallel rank, fixed-order cross-shard reductions, a shared log-softmax scoring tail, and custom differentiation rules around the serving kernels. The authors say the kernel and collective patterns are modular.

In a 200-step FrozenLake test using a public Qwen3-8B checkpoint, Zero-TIM recorded zero mean absolute sampler-trainer logprob difference at every logged step. Standard training and token-level truncated importance sampling kept nonzero differences. The authors report higher observed solve rates for Zero-TIM in these single-seed runs.

## Why it matters if you build agents

If you run your own RL post-training on TPUs, a sampler and trainer that disagree on logprobs can make updates quietly off-policy, so comparing logprobs bitwise across serving and training is a useful diagnostic. The Zero-TIM configuration also preserves sampled token IDs across environment turns, which matters if your agent loop re-tokenizes text between steps. The work is written for JAX on TPU, so teams on other stacks would need their own kernels.
