## Overview

Tool simulation enables controlled agent evaluation by replacing real tool execution with LLM-powered responses. Using the `ToolSimulator` class, you register tools with a decorator, define output schemas, and optionally share state across related tools. When the agent calls a simulated tool, an LLM generates a realistic, schema-validated response instead of executing the real function.

This is useful when:

-   Real tools require live infrastructure (APIs, databases, hardware)
-   You need controllable tool behavior for evaluation
-   You want to test agent tool-use patterns without side effects
-   Tools are still under development or unavailable in the test environment

```python
from typing import Any
from pydantic import BaseModel, Field
from strands import Agent
from strands_evals.simulation.tool_simulator import ToolSimulator

tool_simulator = ToolSimulator()

class WeatherResponse(BaseModel):
    temperature: float = Field(..., description="Temperature in Fahrenheit")
    conditions: str = Field(..., description="Weather conditions")

@tool_simulator.tool(output_schema=WeatherResponse)
def get_weather(city: str) -> dict[str, Any]:
    """Get current weather for a city."""
    pass

weather_tool = tool_simulator.get_tool("get_weather")
agent = Agent(tools=[weather_tool], callback_handler=None)
response = agent("What's the weather in Seattle?")
```

## How It Works

1.  **Tool Registration**: The `@tool_simulator.tool()` decorator captures function metadata (name, docstring, type hints) via Strands’ `FunctionToolMetadata`. The function body is never executed.
2.  **Simulation Wrapper**: When retrieved via `get_tool()`, the real function is replaced with an LLM-backed wrapper that can be passed to a Strands `Agent`.
3.  **LLM Invocation**: On each call, the wrapper builds a prompt containing the tool’s input schema, output schema, user parameters, and current state context, then invokes an Agent to generate a response.
4.  **State Tracking**: A `StateRegistry` records call history and shared state across tools, providing the LLM with context for consistent responses.

## Basic Usage

### Registering a Tool

Define a function with type hints and a docstring, then decorate it with `@tool_simulator.tool()`. Provide an `output_schema` to control the response structure, and the tool can be retrived and passed to a Strands agent.

```python
from typing import Any
from pydantic import BaseModel, Field
from strands import Agent
from strands_evals.simulation.tool_simulator import ToolSimulator

tool_simulator = ToolSimulator()

class OrderStatus(BaseModel):
    order_id: str = Field(..., description="Order identifier")
    status: str = Field(..., description="Current order status")
    estimated_delivery: str = Field(..., description="Estimated delivery date")

@tool_simulator.tool(output_schema=OrderStatus)
def check_order(order_id: str) -> dict[str, Any]:
    """Check the current status of a customer order."""
    pass

order_tool = tool_simulator.get_tool("check_order")
agent = Agent(
    system_prompt="You are a customer service assistant.",
    tools=[order_tool],
    callback_handler=None,
)
response = agent("Where is my order #12345?")
```

### Custom Tool Names

Override the default function name:

```python
@tool_simulator.tool(name="lookup_order", output_schema=OrderStatus)
def check_order(order_id: str) -> dict[str, Any]:
    """Check the current status of a customer order."""
    pass

# Retrieved by custom name
tool = tool_simulator.get_tool("lookup_order")
```

## Shared State

Tools that operate on the same environment can share state via `share_state_id`. When multiple tools share a state key, the LLM sees call history from all of them, enabling consistent behavior across related tools.

```python
from enum import Enum
from pydantic import BaseModel, Field

tool_simulator = ToolSimulator()

class HVACMode(str, Enum):
    HEAT = "heat"
    COOL = "cool"
    AUTO = "auto"
    OFF = "off"

class HVACResponse(BaseModel):
    temperature: float = Field(..., description="Target temperature in Fahrenheit")
    mode: HVACMode = Field(..., description="HVAC mode")
    status: str = Field(default="success", description="Operation status")

class SensorResponse(BaseModel):
    temperature: float = Field(..., description="Current temperature in Fahrenheit")
    humidity: float = Field(..., description="Current humidity percentage")

@tool_simulator.tool(
    share_state_id="room_environment",
    initial_state_description="Room environment: temperature 68F, humidity 45%, HVAC off",
    output_schema=HVACResponse,
)
def hvac_controller(temperature: float, mode: str) -> dict:
    """Control heating/cooling system that affects room temperature and humidity."""
    pass

@tool_simulator.tool(
    share_state_id="room_environment",
    output_schema=SensorResponse,
)
def room_sensor() -> dict:
    """Read current room temperature and humidity."""
    pass

# Both tools share the "room_environment" state
hvac_tool = tool_simulator.get_tool("hvac_controller")
sensor_tool = tool_simulator.get_tool("room_sensor")
agent = Agent(tools=[hvac_tool, sensor_tool], callback_handler=None)
```

### Initial State Description

The `initial_state_description` parameter provides the LLM with baseline context about the environment. This is included in every prompt so the LLM can generate responses consistent with the starting conditions:

```python
@tool_simulator.tool(
    initial_state_description="Database contains users: alice (admin), bob (viewer). No pending invitations.",
    output_schema=UserLookupResponse,
)
def lookup_user(username: str) -> dict:
    """Look up a user in the system."""
    pass
```

## Integration with Experiments

Use ToolSimulator within an Experiment to evaluate agent tool-use behavior end-to-end:

```python
import asyncio

from pydantic import BaseModel, Field
from strands import Agent
from strands_evals import Case, Experiment
from strands_evals.evaluators import GoalSuccessRateEvaluator
from strands_evals.simulation.tool_simulator import ToolSimulator
from strands_evals.mappers import StrandsInMemorySessionMapper
from strands_evals.telemetry import StrandsEvalsTelemetry

# Setup telemetry
telemetry = StrandsEvalsTelemetry().setup_in_memory_exporter()
memory_exporter = telemetry.in_memory_exporter
tool_simulator = ToolSimulator()

class HVACResponse(BaseModel):
    temperature: float = Field(..., description="Target temperature in Fahrenheit")
    mode: str = Field(..., description="HVAC mode")
    status: str = Field(default="success", description="Operation status")

@tool_simulator.tool(
    share_state_id="room_environment",
    initial_state_description="Room: 68F, humidity 45%, HVAC off",
    output_schema=HVACResponse,
)
def hvac_controller(temperature: float, mode: str) -> dict:
    """Control heating/cooling system."""
    pass

def task_function(case: Case) -> dict:
    hvac_tool = tool_simulator.get_tool("hvac_controller")
    agent = Agent(
        trace_attributes={
            "gen_ai.conversation.id": case.session_id,
            "session.id": case.session_id,
        },
        system_prompt="You are an HVAC control assistant.",
        tools=[hvac_tool],
        callback_handler=None,
    )
    response = agent(case.input)

    spans = memory_exporter.get_finished_spans()
    mapper = StrandsInMemorySessionMapper()
    session = mapper.map_to_session(spans, session_id=case.session_id)

    return {"output": str(response), "trajectory": session}

test_cases = [
    Case(name="heat_control", input="Turn on the heat to 72 degrees"),
    Case(name="cool_down", input="It's too hot, cool the room to 65 degrees"),
]

evaluators = [GoalSuccessRateEvaluator()]
experiment = Experiment(cases=test_cases, evaluators=evaluators)

async def main():
    report = await experiment.run_evaluations_async(task_function)
    report.run_display()

asyncio.run(main())
```

## API Reference

### ToolSimulator

| Method | Description |
| --- | --- |
| `tool(output_schema, name, share_state_id, initial_state_description)` | Decorator to register a simulated tool |
| `get_tool(tool_name)` | Retrieve a simulation-wrapped tool by name |
| `get_state(state_key)` | Get current state for a tool or shared state group |
| `list_tools()` | List all registered tool names |
| `clear_tools()` | Clear all registered tools |

### StateRegistry

| Method | Description |
| --- | --- |
| `initialize_state_via_description(description, state_key)` | Pre-seed state with context |
| `get_state(state_key)` | Retrieve state dict for a tool or shared group |
| `cache_tool_call(tool_name, state_key, response_data, parameters)` | Record a tool call |
| `clear_state(state_key)` | Clear state for a specific key |

### Data Models

**RegisteredTool:**

```python
class RegisteredTool(BaseModel):
    name: str                                    # Tool name
    function: Callable | None                    # Underlying DecoratedFunctionTool
    output_schema: type[BaseModel] | None        # Pydantic output schema
    initial_state_description: str | None         # Initial state context
    share_state_id: str | None                   # Shared state key
```

**DefaultToolResponse:**

```python
class DefaultToolResponse(BaseModel):
    response: str  # Default response when no output_schema is provided
```

## Advanced Usage and Configurations

### Inspecting State

Use `get_state()` to examine call history and initial state for debugging:

```python
# Before agent invocation
initial_state = tool_simulator.get_state("room_environment")
print(f"Initial state: {initial_state.get('initial_state')}")
print(f"Previous calls: {initial_state.get('previous_calls', [])}")

# After agent invocation
final_state = tool_simulator.get_state("room_environment")
for call in final_state["previous_calls"]:
    print(f"  {call['tool_name']}: {call['parameters']} -> {call['response']}")
```

Each call record contains:

-   `tool_name`: Name of the tool that was called
-   `parameters`: The parameters passed to the tool
-   `response`: The LLM-generated response
-   `timestamp`: When the call was made

### Configuration

#### Custom Model

Specify a different model for simulation inference:

```python
# Via model ID string (Bedrock)
tool_simulator = ToolSimulator(model="anthropic.claude-haiku-4-5-20251001-v1:0")

# Via Strands Model provider
from strands.models import BedrockModel

model = BedrockModel(model_id="anthropic.claude-haiku-4-5-20251001-v1:0")
tool_simulator = ToolSimulator(model=model)
```

#### Cache Size

Control how many tool calls are retained per state key:

```python
# Default: 20 calls per state key
tool_simulator = ToolSimulator(max_tool_call_cache_size=20)

# Increase for long-running evaluations
tool_simulator = ToolSimulator(max_tool_call_cache_size=50)
```

When the cache is full, the oldest calls are evicted (FIFO).

#### Custom State Registry

Provide your own `StateRegistry` for advanced state management:

```python
from strands_evals.simulation.tool_simulator import StateRegistry, ToolSimulator

registry = StateRegistry(max_tool_call_cache_size=100)
tool_simulator = ToolSimulator(state_registry=registry)
```

### Running Independent Simulator Instances

You can create multiple `ToolSimulator` instances side by side. Each instance maintains its own tool registry and state, so you can run parallel experiment configurations in the same codebase:

```python
simulator_a = ToolSimulator()
simulator_b = ToolSimulator()

# Each instance has an independent tool registry and state --
# ideal for comparing agent behavior across different tool setups.
```

This is useful when you want to A/B test different tool configurations, output schemas, or initial state descriptions against the same agent.

### Seeding State from Real Data

Because `initial_state_description` accepts natural language, you can get creative with how you seed context. For tools that interact with tabular data, use a `DataFrame.describe()` call to generate statistical summaries and pass those statistics directly as the state description. `ToolSimulator` will generate responses that reflect realistic data distributions, without ever accessing the actual data:

```python
import pandas as pd

df = pd.read_csv("sales_data.csv")
stats_summary = df.describe().to_string()

@tool_simulator.tool(
    initial_state_description=f"Sales database statistics:\n{stats_summary}",
    output_schema=SalesQueryResponse,
)
def query_sales(region: str, quarter: str) -> dict:
    """Query sales data by region and quarter."""
    pass
```

This approach lets you ground simulated responses in real data characteristics while keeping the actual data out of the evaluation loop.

## Troubleshooting

### Issue: Tool Not Found

`get_tool()` returns `None` if the tool name doesn’t match:

```python
tool = tool_simulator.get_tool("my_tool")
if tool is None:
    print(f"Available tools: {tool_simulator.list_tools()}")
```

### Issue: Inconsistent Responses Across Calls

Ensure related tools share state and that initial state is set:

```python
# Without shared state, each tool has independent context
@tool_simulator.tool(share_state_id="shared_env", initial_state_description="...", output_schema=...)
def tool_a(...): ...

@tool_simulator.tool(share_state_id="shared_env", output_schema=...)
def tool_b(...): ...
```

### Issue: State Re-initialization Warning

If you see a warning about state already being initialized, it means two tools with the same `share_state_id` both provide `initial_state_description`. Only the first one takes effect:

```python
# First tool initializes state
@tool_simulator.tool(
    share_state_id="env",
    initial_state_description="Starting state",  # This takes effect
    output_schema=...,
)
def tool_a(...): ...

# Second tool's initial_state_description is ignored with a warning
@tool_simulator.tool(
    share_state_id="env",
    initial_state_description="Different state",  # Ignored
    output_schema=...,
)
def tool_b(...): ...
```

## Related Documentation

-   [Simulators Overview](/docs/user-guide/evals-sdk/simulators/index.md): Overview of the simulator framework
-   [User Simulation](/docs/user-guide/evals-sdk/simulators/user_simulation/index.md): Simulate multi-turn user conversations
-   [Quickstart Guide](/docs/user-guide/evals-sdk/quickstart/index.md): Get started with Strands Evals
-   [Goal Success Rate Evaluator](/docs/user-guide/evals-sdk/evaluators/goal_success_rate_evaluator/index.md): Assess goal completion

## Related pages

- [Failure Communication Evaluator](/docs/user-guide/evals-sdk/evaluators/failure_communication_evaluator/index.md) (2 shared tags)
- [Partial Completion Evaluator](/docs/user-guide/evals-sdk/evaluators/partial_completion_evaluator/index.md) (2 shared tags)
- [Recovery Strategy Evaluator](/docs/user-guide/evals-sdk/evaluators/recovery_strategy_evaluator/index.md) (2 shared tags)
- [Deterministic Evaluators](/docs/user-guide/evals-sdk/evaluators/deterministic_evaluators/index.md) (1 shared tag)
- [Experiment Generator](/docs/user-guide/evals-sdk/experiment_generator/index.md) (1 shared tag)
- [Simulators](/docs/user-guide/evals-sdk/simulators/index.md) (1 shared tag)
- [Tool Parameter Accuracy Evaluator](/docs/user-guide/evals-sdk/evaluators/tool_parameter_evaluator/index.md) (1 shared tag)
- [Tool Selection Accuracy Evaluator](/docs/user-guide/evals-sdk/evaluators/tool_selection_evaluator/index.md) (1 shared tag)
- [Trajectory Evaluator](/docs/user-guide/evals-sdk/evaluators/trajectory_evaluator/index.md) (1 shared tag)
- [User Simulation](/docs/user-guide/evals-sdk/simulators/user_simulation/index.md) (1 shared tag)