The model has 120B total parameters with 12B active per token, pairing a hybrid Mixture-of-Experts language backbone with a C-RADIO vision encoder. The architecture supports up to 1 million tokens of context, but the JumpStart serving configurations cap context between 520 and 15,600 tokens. Reasoning is on by default and can be turned off per request.
JumpStart deploys the model from SageMaker Studio, with the SageMaker Python SDK v3 (3.25 or later), or from the Hugging Face model page. New accounts start with zero quota for ml.g7 instances, so they must request an increase through Service Quotas before deploying. The NVFP4 variant defaults to ml.g7.24xlarge, and the BF16 variant runs on ml.p5en.48xlarge.
In AWS's test with five concurrent streaming requests, the NVFP4 variant on ml.g7.24xlarge averaged 63.07 output tokens per second per request, with 0.45 seconds to first token. The BF16 variant on ml.p5en.48xlarge averaged 140.85 tokens per second in the same test. Each request accepts one image and one video, and PDF pages must be rendered to images first.
Why it matters if you build agents
Developers building computer-use, GUI or document agents can host a vision-language model in their own AWS account, with tool calling and JSON schema output available. Check the serving context limit, the one-image-and-one-video-per-request rule, and the need to set max_tokens to at least 8,192 when video reasoning is enabled. Budget time for the Service Quotas increase, since new accounts cannot launch G7 endpoints until it is approved.
Get stories like this every weekday
One short email with the day's agent tooling news.