AI Agent News/stories/amazon-bedrock-adds-explicit-prompt-caching-for-open-weight-models.md

Infrastructure

Amazon Bedrock adds explicit prompt caching for open-weight models

Open-weight models on Amazon Bedrock now support explicit prompt caching alongside implicit caching, according to an AWS Machine Learning blog post. The post shows how to read cache usage from Chat Completions responses and how an OpenCode coding agent session benefits from implicit caching.

Source: aws.amazon.com/blogs/machine-learning/use-prompt-caching-with-open-weight-models-on-amazon-bedrock

Prompt caching lets Amazon Bedrock reuse the computed state of a matching prompt prefix, so repeated context such as a system prompt, reference documents, tool definitions or conversation history is read from cache rather than processed again. Depending on the model, implicit caching happens automatically, explicit caching is set by marking a breakpoint, or both can be used. Cache reads are billed at a lower rate than standard input, while cache writes can cost more on some models.

Explicit mode is enabled with prompt_cache_options set to explicit on the Chat Completions API, and a prompt_cache_breakpoint marks the end of the reusable prefix. Content after the last breakpoint is billed at the standard input rate. The post's Python samples use the OpenAI SDK pointed at the Bedrock endpoint, with short-term API keys generated from AWS credentials.

For coding agents, the post applies implicit caching to OpenCode 1.18.35 configured with a custom OpenAI-compatible provider. Each request in a turn can read the earlier prefix from cache and write only the new tail, and no caching configuration is needed. The post also lists best practices such as placing static content first and tracking the CacheReadInputTokenCount and CacheWriteInputTokenCount metrics.

Why it matters if you build agents

If your agents resend a long system prompt, tool definitions or conversation history on each step, implicit caching may already reduce cost and latency with no code changes.

Use explicit caching when a fixed prefix is followed by a question that changes each call, so that cache writes are limited to the shared prefix and you do not pay write rates for content that is never read back.

Check the usage fields on each response and the model card's minimum prefix length, breakpoint limit and TTL, since a prefix that is too short or an idle gap longer than the TTL will not produce cache hits.

Get stories like this every weekday

One short email with the day's agent tooling news.

Subscribe