feat: Enhance error messages for tool and agent not found errors

Merge https://github.com/google/adk-python/pull/3219

## Summary

Enhance error messages for tool and agent not found errors to provide actionable guidance and reduce developer debugging time from hours to minutes.

Fixes #3217

## Changes

### Modified Files

1. **`src/google/adk/flows/llm_flows/functions.py`**
   - Enhanced `_get_tool()` error message with:
     - Available tools list (formatted, truncated to 20 for readability)
     - Possible causes
     - Suggested fixes
     - Fuzzy matching suggestions

2. **`src/google/adk/agents/llm_agent.py`**
   - Enhanced `__get_agent_to_run()` error message with:
     - Available agents list (formatted, truncated to 20 for readability)
     - Timing/ordering issue explanation
     - Fuzzy matching for agent names
   - Added `_get_available_agent_names()` helper method

### New Test Files

3. **`tests/unittests/flows/llm_flows/test_functions_error_messages.py`**
   - Tests for enhanced tool not found error messages
   - Fuzzy matching validation
   - Edge cases (no close matches, empty tools dict, 100+ tools)

4. **`tests/unittests/agents/test_llm_agent_error_messages.py`**
   - Tests for enhanced agent not found error messages
   - Agent tree traversal validation
   - Fuzzy matching for agents
   - Long list truncation

## Testing Plan

### Unit Tests

```bash
pytest tests/unittests/flows/llm_flows/test_functions_error_messages.py -v
pytest tests/unittests/agents/test_llm_agent_error_messages.py -v
```

**Results**:  8/8 tests passing

```
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_enhanced_error PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_fuzzy_matching PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_no_fuzzy_match PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_truncates_long_list PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_enhanced_error PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_fuzzy_matching PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_tree_traversal PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_truncates_long_list PASSED

8 passed, 1 warning in 4.38s
```

### Example Enhanced Error Messages

#### Before (Current Error)

```
ValueError: Function get_equipment_specs is not found in the tools_dict: dict_keys(['get_equipment_details', 'query_vendor_catalog', 'score_proposals'])
```

#### After (Enhanced Error)

```
Function 'get_equipment_specs' is not found in available tools.

Available tools: get_equipment_details, query_vendor_catalog, score_proposals

Possible causes:
  1. LLM hallucinated the function name - review agent instruction clarity
  2. Tool not registered - verify agent.tools list
  3. Name mismatch - check for typos

Suggested fixes:
  - Review agent instruction to ensure tool usage is clear
  - Verify tool is included in agent.tools list
  - Check for typos in function name

Did you mean one of these?
  - get_equipment_details
```

## Community Impact

- **Addresses 3 active issues**: #2050, #2933 (12 comments), #2164
- **Reduces debugging time** from 3+ hours to < 5 minutes (validated in production multi-agent RFQ solution for recent partner nanothon initiative)
- **Improves developer experience** for new ADK users

## Implementation Details

- Uses standard library `difflib` for fuzzy matching (no new dependencies)
- Error path only (no performance impact on happy path)
- Measured performance: < 0.03ms per error
- Truncates long lists to first 20 items to prevent log overflow
- Fully backward compatible (same exception types)

## Checklist

- [x] Unit tests added and passing (8/8 tests)
- [x] Code formatted with `./autoformat.sh` (isort + pyink)
- [x] No new dependencies (uses standard library `difflib`)
- [x] Docstrings updated
- [x] Tested with Python 3.11
- [x] Issue #3217 created and linked

## Related Issues

- Fixes #3217
- Addresses #2050 - Tool verification callback request
- Addresses #2933 - How to handle "Function is not found in the tools_dict" Error
- Addresses #2164 - ValueError: {agent} not found in agent tree

---

**Note**: For production scenarios where LLM tool hallucinations occur, ADK's built-in [`ReflectAndRetryToolPlugin`](https://github.com/google/adk-python/blob/main/src/google/adk/plugins/reflect_retry_tool_plugin.py) can automatically retry failed tool calls (available since v1.16.0). This PR's enhanced error messages complement that by helping developers quickly identify and fix configuration issues during development.

Cheers, JP

Co-authored-by: Yvonne Yu <yyyu@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3219 from jpantsjoha:feat/better-error-messages a4df8bfb031685dce9e528d8eb7006f53447b75b
PiperOrigin-RevId: 826132579
This commit is contained in:
Jaroslav Pantsjoha
2025-10-30 12:08:13 -07:00
committed by Copybara-Service
parent 5eca72f9bf
commit 34d9b53f37
4 changed files with 222 additions and 4 deletions
@@ -0,0 +1,88 @@
# Copyright 2025 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests for enhanced error messages in function tool handling."""
from google.adk.flows.llm_flows.functions import _get_tool
from google.adk.tools import BaseTool
from google.genai import types
import pytest
# Mock tool for testing error messages
class MockTool(BaseTool):
"""Mock tool for testing error messages."""
def __init__(self, name: str = 'mock_tool'):
super().__init__(name=name, description=f'Mock tool: {name}')
def call(self, *args, **kwargs):
return 'mock_response'
def test_tool_not_found_enhanced_error():
"""Verify enhanced error message for tool not found."""
function_call = types.FunctionCall(name='nonexistent_tool', args={})
tools_dict = {
'get_weather': MockTool(name='get_weather'),
'calculate_sum': MockTool(name='calculate_sum'),
'search_database': MockTool(name='search_database'),
}
with pytest.raises(ValueError) as exc_info:
_get_tool(function_call, tools_dict)
error_msg = str(exc_info.value)
# Verify error message components
assert 'nonexistent_tool' in error_msg
assert 'Available tools:' in error_msg
assert 'get_weather' in error_msg
assert 'Possible causes:' in error_msg
assert 'Suggested fixes:' in error_msg
def test_tool_not_found_with_different_name():
"""Verify error message contains basic information."""
function_call = types.FunctionCall(name='completely_different', args={})
tools_dict = {
'get_weather': MockTool(name='get_weather'),
'calculate_sum': MockTool(name='calculate_sum'),
}
with pytest.raises(ValueError) as exc_info:
_get_tool(function_call, tools_dict)
error_msg = str(exc_info.value)
# Verify error message contains basic information
assert 'completely_different' in error_msg
assert 'Available tools:' in error_msg
def test_tool_not_found_shows_all_tools():
"""Verify error message shows all tools (no truncation)."""
function_call = types.FunctionCall(name='nonexistent', args={})
# Create 100 tools
tools_dict = {f'tool_{i}': MockTool(name=f'tool_{i}') for i in range(100)}
with pytest.raises(ValueError) as exc_info:
_get_tool(function_call, tools_dict)
error_msg = str(exc_info.value)
# Verify all tools are shown (no truncation)
assert 'tool_0' in error_msg # First tool shown
assert 'tool_99' in error_msg # Last tool also shown
assert 'showing first 20 of' not in error_msg # No truncation message