Context Estimation

Experimental Feature

This feature is experimental and may change in future versions. Use with caution in production environments.

Experimental

The Context Manager strategy API is experimental and may change in future versions.

The SDK estimates how much of the model’s context window is in use so it can trigger compression before overflow. Three pieces work together: the context window limit, token counting, and utilization estimation.

Context window limit

The threshold check requires the model’s context window size. The SDK auto-populates context_window_limitcontextWindowLimit from built-in lookup tables for known models.

Python

Override it manually for models not in the lookup table:

model = BedrockModel(
    model_id="my-custom-model",
    context_window_limit=128_000,
)

TypeScript

Override it manually for models not in the lookup table:

import { BedrockModel } from '@strands-agents/sdk'

const model = new BedrockModel({
  modelId: 'my-custom-model',
  contextWindowLimit: 128_000,
})

Inaccurate compression with default fallback

If context_window_limitcontextWindowLimit is not set and the model ID is not in the built-in lookup table, the SDK falls back to a default of 200,000 tokens. When the actual context window is significantly different, compression thresholds will not behave correctly.

Token estimation

The agent estimates input tokens using the following strategy:

1. Known baseline: reads input_tokens + output_tokensinputTokens + outputTokens from the last assistant message’s usage metadata
2. Delta estimation: estimates tokens for new messages added since that baseline using the model’s count_tokens()countTokens() method
3. Cold start fallback: when no prior usage metadata exists (first call or after session restore), estimates all messages via count_tokens()countTokens()

The count_tokens()countTokens() method uses a character-based heuristic by default (characters / 4 for text, characters / 2 for JSON). Some model providers support native token counting APIs for exact counts. For an example in both SDKs, see Amazon Bedrock token counting.

Utilization estimation

The Model base class provides an estimate_utilization(input_tokens)estimateUtilization(inputTokens) method that computes the fraction of the context window consumed by a given input token count:

Python

ratio = model.estimate_utilization(
    input_tokens=projected_tokens,
)
# ratio is 0-1+ (above 1.0 means overflow)

TypeScript

import { BedrockModel } from '@strands-agents/sdk'

const ratio = model.estimateUtilization(projectedTokens)
// ratio is 0-1+ (above 1.0 means overflow)

The method divides input_tokensinputTokens by the model’s context_window_limitcontextWindowLimit, falling back to the 200,000 default with a warning when not configured. A return value above 1.0 means overflow.

Both the built-in context management modes and the conversation managers use this internally for compression decisions. You can also call it directly when building custom logic.

Tags

conversation-management17token-management8