This change removes the `evaluate`, `_evaluate_row`, `are_tools_equal`, `_remove_tool_outputs`, `_report_failures`, and `_print_results` static methods from `TrajectoryEvaluator`, along with their corresponding unit tests. These methods were previously marked as deprecated.
PiperOrigin-RevId: 817477494
This CL updates the "What's new" section to include Resumability, ReflectRetryToolPlugin, Context compaction, and Search tool support. It also moves "Agent Config" and "Tool Confirmation" from "What's new" to "Key Features".
PiperOrigin-RevId: 817469210
The added section provides details for the community call on Oct 15, 2025, including the agenda and links to join and add to calendars.
PiperOrigin-RevId: 817457276
Agent developers can now create an eval set and add eval cases through command line itself. Adding an eval case is limited only to specifying conversation scenarios.
Sample comamnds:
- Create an eval set:
adk eval_set create \
contributing/samples/hello_world \
set_01
- Add an eval case with scenario file
Content of scenarios.json file:
'{"scenarios": [{"starting_prompt": "hello", "conversation_plan": "world"}]}'
adk eval_set add_eval_case \
contributing/samples/hello_world \
set_01 \
--scenarios scenarios.json
PiperOrigin-RevId: 817456117
The `agent_loader.load_agent` method can now return an `App` object. This change unwraps the `App` to get its `root_agent` before passing it to the graph builder, makes sure a `BaseAgent` instance is always used
PiperOrigin-RevId: 817209601
Details:
- Introduces a concept of `ConversationScenario` to represent a scenario that user simulator is supposed to follow.
- Introduces a `UserSimulator` interface, that one should implement. UserSimulator interface will be integrated with LocalEvalService in subsequent PRs.
PiperOrigin-RevId: 816883699
When there are multiple intervals and compactions, the original implementation only keep the last one. The right implementation is to keep as many compaction events/summary as the requested internals.
PiperOrigin-RevId: 816516662
This plugin intercepts tool failures, provides structured guidance to the LLM for reflection and correction, and retries the operation up to a configurable limit.
**Key Features:**
- **Concurrency Safe:** Uses locking to safely handle parallel tool
executions
- **Configurable Scope:** Tracks failures per-invocation (default) or globally
using the `TrackingScope` enum.
- **Extensible Scoping:** The `_get_scope_key` method can be overridden to
implement custom tracking logic (e.g., per-user or per-session).
- **Granular Tracking:** Failure counts are tracked per-tool within the
defined scope. A success with one tool resets its counter without affecting
others.
- **Custom Error Extraction:** Supports detecting errors in normal tool
responses
that
don't throw exceptions, by overriding the `extract_error_from_result`
method.
**Example:**
```python
from my_project.plugins import ReflectAndRetryToolPlugin, TrackingScope
# Example 1: (MOST COMMON USAGE):
# Track failures only within the current agent invocation (default).
error_handling_plugin = ReflectAndRetryToolPlugin(max_retries=3)
# Example 2:
# Track failures globally across all turns and users.
global_error_handling_plugin = ReflectAndRetryToolPlugin(max_retries=5,
scope=TrackingScope.GLOBAL)
# Example 3:
# Retry on failures but do not throw exceptions.
error_handling_plugin =
ReflectAndRetryToolPlugin(max_retries=3,
throw_exception_if_retry_exceeded=False)
# Example 4:
# Track failures in successful tool responses that contain errors.
class CustomRetryPlugin(ReflectAndRetryToolPlugin):
async def extract_error_from_result(self, *, tool, tool_args,tool_context,
result):
# Detect error based on response content
if result.get('status') == 'error':
return result
return None # No error detected
error_handling_plugin = CustomRetryPlugin(max_retries=5)
```
PiperOrigin-RevId: 816456549
