## Overview

Deterministic evaluators provide fast, code-based evaluation without LLM judges. They perform exact checks on outputs, trajectories, and environment state — making them ideal for regression testing, CI/CD pipelines, and cases where evaluation criteria are objective and well-defined.

## Key Features

-   **No LLM Required**: Pure code-based evaluation — fast and free
-   **Deterministic Results**: Same input always produces the same score
-   **Multiple Check Types**: Output matching, tool call verification, and state comparison
-   **Async Support**: All evaluators support both sync and async evaluation

## Available Evaluators

### Output Evaluators

#### `Equals`

Checks if `actual_output` equals an expected value.

```python
from strands_evals.evaluators import Equals

# Compare against explicit value
evaluator = Equals(value="Paris")

# Or compare against case's expected_output (when value is None)
evaluator = Equals()
```

**Parameters:**

-   `value` (optional): The expected value. If `None`, uses `expected_output` from the evaluation case.

#### `Contains`

Checks if `actual_output` contains a substring.

```python
from strands_evals.evaluators import Contains

evaluator = Contains(value="Paris", case_sensitive=False)
```

**Parameters:**

-   `value` (required): The substring to search for.
-   `case_sensitive` (optional, default `True`): Whether the check is case-sensitive.

#### `StartsWith`

Checks if `actual_output` starts with a prefix.

```python
from strands_evals.evaluators import StartsWith

evaluator = StartsWith(value="The capital", case_sensitive=False)
```

**Parameters:**

-   `value` (required): The prefix to check.
-   `case_sensitive` (optional, default `True`): Whether the check is case-sensitive.

### Trajectory Evaluators

#### `ToolCalled`

Checks if a specific tool was called in the trajectory. Works with both list-based trajectories and `Session` objects.

```python
from strands_evals.evaluators import ToolCalled

evaluator = ToolCalled(tool_name="calculator")
```

**Parameters:**

-   `tool_name` (required): Name of the tool to check for.

### Environment State Evaluators

#### `StateEquals`

Checks if a named environment state matches an expected value. Useful for verifying that tool-using agents produce the correct side effects.

```python
from strands_evals.evaluators import StateEquals

# Compare against explicit value
evaluator = StateEquals(name="temperature", value=72.0)

# Or compare against case's expected_environment_state
evaluator = StateEquals(name="temperature")
```

**Parameters:**

-   `name` (required): Name of the environment state to check.
-   `value` (optional): Expected value. If `None`, uses `expected_environment_state` from the evaluation case.

## Usage Example

```python
import asyncio

from strands import Agent
from strands_evals import Case, Experiment
from strands_evals.evaluators import Equals, Contains, ToolCalled

cases = [
    Case(
        name="capital-check",
        input="What is the capital of France?",
        expected_output="Paris"
    )
]

# Combine multiple deterministic checks
evaluators = [
    Contains(value="Paris", case_sensitive=False),
    Contains(value="France", case_sensitive=False),
]

def get_response(case: Case) -> str:
    agent = Agent(callback_handler=None)
    return str(agent(case.input))

experiment = Experiment(cases=cases, evaluators=evaluators)

async def main():
    report = await experiment.run_evaluations_async(get_response)
    report.run_display()

asyncio.run(main())
```

## Combining with LLM Evaluators

Deterministic evaluators work well alongside LLM-based evaluators for comprehensive assessment:

```python
from strands_evals.evaluators import Contains, HelpfulnessEvaluator, CorrectnessEvaluator

evaluators = [
    Contains(value="Paris"),              # Fast deterministic check
    CorrectnessEvaluator(),               # LLM-based correctness
    HelpfulnessEvaluator(),               # LLM-based helpfulness
]
```

## Best Practices

1.  **Use for regression testing**: Deterministic evaluators are ideal for CI/CD since they’re fast and don’t require API calls
2.  **Combine with LLM evaluators**: Use deterministic checks as a first pass, then LLM evaluators for nuanced assessment
3.  **Case sensitivity**: Use `case_sensitive=False` when exact casing doesn’t matter
4.  **State verification**: Use `StateEquals` when your agent modifies external state through tools

## Related Evaluators

-   [**OutputEvaluator**](/docs/user-guide/evals-sdk/evaluators/output_evaluator/index.md): LLM-based output evaluation with custom rubrics
-   [**TrajectoryEvaluator**](/docs/user-guide/evals-sdk/evaluators/trajectory_evaluator/index.md): LLM-based trajectory evaluation
-   [**CustomEvaluator**](/docs/user-guide/evals-sdk/evaluators/custom_evaluator/index.md): Build your own evaluation logic

## Related pages

- [Tool Parameter Accuracy Evaluator](/docs/user-guide/evals-sdk/evaluators/tool_parameter_evaluator/index.md) (1 shared tag)
- [Tool Selection Accuracy Evaluator](/docs/user-guide/evals-sdk/evaluators/tool_selection_evaluator/index.md) (1 shared tag)
- [Trajectory Evaluator](/docs/user-guide/evals-sdk/evaluators/trajectory_evaluator/index.md) (1 shared tag)
- [Tool Simulation](/docs/user-guide/evals-sdk/simulators/tool_simulation/index.md) (1 shared tag)
- [Failure Communication Evaluator](/docs/user-guide/evals-sdk/evaluators/failure_communication_evaluator/index.md) (1 shared tag)
- [Partial Completion Evaluator](/docs/user-guide/evals-sdk/evaluators/partial_completion_evaluator/index.md) (1 shared tag)
- [Recovery Strategy Evaluator](/docs/user-guide/evals-sdk/evaluators/recovery_strategy_evaluator/index.md) (1 shared tag)