AWS publishes guide to deploying NVIDIA Nemotron 3.5 Super VL on EKS

NVIDIA Nemotron 3.5 Super VL is an open multimodal model distilled from Nemotron 3 Ultra, with 120B total parameters and 12B active per token. It combines a hybrid Mamba-2, mixture-of-experts and attention design with a RADIO vision encoder, supports context lengths up to 1M tokens, and takes text, image, document and video input while producing text output. NVIDIA publishes the weights, training datasets and post-training recipes.

The reference architecture uses the same Kubernetes manifests on standard EKS, EKS Auto Mode and SageMaker HyperPod EKS. Container images are stored in Amazon ECR, model weights sit on a shared FSx for Lustre volume, and EFA connects the GPU nodes. NVIDIA Dynamo serves the model either aggregated on one node or disaggregated across nodes, and the setup was validated on 8-GPU B200 and H200 instances.

Validation uses an NCCL all-reduce micro-benchmark to confirm GPU and EFA traffic, and the aiperf tool measures time to first token, inter-token latency and tokens per second at different concurrency levels. In the example sweep, aggregated and disaggregated two-node deployments showed similar ITL but different TTFT. The full deployment walkthrough is in the aws-do-eks project.

Teams that want a self-hosted multimodal model for coding, research or document agents now have a tested EKS path with published minimum GPU counts, such as 8 B200 GPUs for BF16 or 2 for NVFP4 under vLLM. Measuring TTFT and ITL across concurrency levels, as the post describes, gives a basis for sizing prefill and decode capacity against a latency budget.