Together AI unveils Together Compiler for self-tuning open model inference

Together AI says open-source models are changing faster, using more varied architectures, with traffic patterns that shift and hardware that is replaced more often. The company argues that static, hand-tuned inference configurations and kernels built for one setting fall behind in this environment, and it calls its response Polymorphic Inference.

Together Compiler, the first project under this approach, treats inference optimization as an agentic search problem, pairing a swarm of agents with a persistent engine for fast iterations. The company says it already runs these optimization loops on some of its most popular serverless endpoints, where it found further gains on code that was already highly optimized.

Together says self-serve Continuous Compilation for dedicated endpoints is coming, and Together Compiler is available in limited beta, with early access on request. The post also previews work on shape-shifting speculative decoding, a ThunderAgent Router for routing across heterogeneous deployments, Together Post-Training, and TogetherLink.

Developers running open models on Together endpoints may see performance adjust as their traffic changes, without hand-writing new kernels or engine configs. Teams with dedicated endpoints should watch for the self-serve beta, which would let them tune optimization for their own workloads. The performance claims are the company's own, and the post does not include benchmark figures in the text.