Commit Graph

557 Commits

Author SHA1 Message Date
Jaroslav Pantsjoha 34d9b53f37 feat: Enhance error messages for tool and agent not found errors
Merge https://github.com/google/adk-python/pull/3219

## Summary

Enhance error messages for tool and agent not found errors to provide actionable guidance and reduce developer debugging time from hours to minutes.

Fixes #3217

## Changes

### Modified Files

1. **`src/google/adk/flows/llm_flows/functions.py`**
   - Enhanced `_get_tool()` error message with:
     - Available tools list (formatted, truncated to 20 for readability)
     - Possible causes
     - Suggested fixes
     - Fuzzy matching suggestions

2. **`src/google/adk/agents/llm_agent.py`**
   - Enhanced `__get_agent_to_run()` error message with:
     - Available agents list (formatted, truncated to 20 for readability)
     - Timing/ordering issue explanation
     - Fuzzy matching for agent names
   - Added `_get_available_agent_names()` helper method

### New Test Files

3. **`tests/unittests/flows/llm_flows/test_functions_error_messages.py`**
   - Tests for enhanced tool not found error messages
   - Fuzzy matching validation
   - Edge cases (no close matches, empty tools dict, 100+ tools)

4. **`tests/unittests/agents/test_llm_agent_error_messages.py`**
   - Tests for enhanced agent not found error messages
   - Agent tree traversal validation
   - Fuzzy matching for agents
   - Long list truncation

## Testing Plan

### Unit Tests

```bash
pytest tests/unittests/flows/llm_flows/test_functions_error_messages.py -v
pytest tests/unittests/agents/test_llm_agent_error_messages.py -v
```

**Results**:  8/8 tests passing

```
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_enhanced_error PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_fuzzy_matching PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_no_fuzzy_match PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_truncates_long_list PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_enhanced_error PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_fuzzy_matching PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_tree_traversal PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_truncates_long_list PASSED

8 passed, 1 warning in 4.38s
```

### Example Enhanced Error Messages

#### Before (Current Error)

```
ValueError: Function get_equipment_specs is not found in the tools_dict: dict_keys(['get_equipment_details', 'query_vendor_catalog', 'score_proposals'])
```

#### After (Enhanced Error)

```
Function 'get_equipment_specs' is not found in available tools.

Available tools: get_equipment_details, query_vendor_catalog, score_proposals

Possible causes:
  1. LLM hallucinated the function name - review agent instruction clarity
  2. Tool not registered - verify agent.tools list
  3. Name mismatch - check for typos

Suggested fixes:
  - Review agent instruction to ensure tool usage is clear
  - Verify tool is included in agent.tools list
  - Check for typos in function name

Did you mean one of these?
  - get_equipment_details
```

## Community Impact

- **Addresses 3 active issues**: #2050, #2933 (12 comments), #2164
- **Reduces debugging time** from 3+ hours to < 5 minutes (validated in production multi-agent RFQ solution for recent partner nanothon initiative)
- **Improves developer experience** for new ADK users

## Implementation Details

- Uses standard library `difflib` for fuzzy matching (no new dependencies)
- Error path only (no performance impact on happy path)
- Measured performance: < 0.03ms per error
- Truncates long lists to first 20 items to prevent log overflow
- Fully backward compatible (same exception types)

## Checklist

- [x] Unit tests added and passing (8/8 tests)
- [x] Code formatted with `./autoformat.sh` (isort + pyink)
- [x] No new dependencies (uses standard library `difflib`)
- [x] Docstrings updated
- [x] Tested with Python 3.11
- [x] Issue #3217 created and linked

## Related Issues

- Fixes #3217
- Addresses #2050 - Tool verification callback request
- Addresses #2933 - How to handle "Function is not found in the tools_dict" Error
- Addresses #2164 - ValueError: {agent} not found in agent tree

---

**Note**: For production scenarios where LLM tool hallucinations occur, ADK's built-in [`ReflectAndRetryToolPlugin`](https://github.com/google/adk-python/blob/main/src/google/adk/plugins/reflect_retry_tool_plugin.py) can automatically retry failed tool calls (available since v1.16.0). This PR's enhanced error messages complement that by helping developers quickly identify and fix configuration issues during development.

Cheers, JP

Co-authored-by: Yvonne Yu <yyyu@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3219 from jpantsjoha:feat/better-error-messages a4df8bfb031685dce9e528d8eb7006f53447b75b
PiperOrigin-RevId: 826132579
2025-10-30 12:08:13 -07:00
Ayush Agrawal 5eca72f9bf fix: Correct message part ordering in A2A history
Merges test case from https://github.com/google/adk-python/pull/3262

Fixes: #3260

Co-authored-by: Ayush Agrawal <ayushagra@google.com>
PiperOrigin-RevId: 826119626
2025-10-30 11:36:38 -07:00
George Weale 1e6a9daa63 fix: Change instruction insertion to respect tool call/response pairs
Make sure _add_instructions_to_user_content skips over user messages that carry function_response parts so tool_use/tool_result blocks stay together

Close #3229

Co-authored-by: George Weale <gweale@google.com>
PiperOrigin-RevId: 826076141
2025-10-30 09:54:49 -07:00
Ieva Grublyte c0892c725c chore: remove legacy validation that forbids co-exist of agent transfer and output_schema. Closes #3318
Remove validation for output_schema and agent transfer flags.

The check that prevented `output_schema` from co-existing with agent transfer capabilities (`disallow_transfer_to_parent` or `disallow_transfer_to_peers` being False) has been removed. The agent will no longer automatically set these transfer flags to True when `output_schema` is present.

Co-authored-by: Ieva Grublyte <ievagrublyte@google.com>
PiperOrigin-RevId: 825998224
2025-10-30 06:06:26 -07:00
Ayam ce8f674a28 fix: Allow LLM request to override the model used in the generate content async method in LiteLLM
Merge https://github.com/google/adk-python/pull/3066

Close #3065

Co-authored-by: Raman Mangla <ramanmangla@google.com>
PiperOrigin-RevId: 825880794
2025-10-29 23:34:33 -07:00
hung12ct fc15c9a0c3 fix: Update DynamicPickleType to support MySQL dialect
Merge https://github.com/google/adk-python/pull/3282

The `process_bind_param` and `process_result_value` methods in the `DynamicPickleType` class have been modified to handle MySQL dialect in addition to Spanner. This change ensures that pickled values are correctly processed for both database types.

**Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.**

### Link to Issue or Description of Change

**1. Link to an existing issue (if applicable):**

- Closes: #3283

**2. Or, if no issue exists, describe the change:**

_If applicable, please follow the issue templates to provide as much detail as
possible._

**Problem:**
When using `DatabaseSessionService` with MySQL backend in google-adk v1.17.0, the application crashes with the following error: app.resources.runner:event_generator:260 - Error in event_generator: (builtins.TypeError) 'tuple' object cannot be interpreted as an integer
<img width="1237" height="129" alt="image" src="https://github.com/user-attachments/assets/0a5fc223-600a-4a92-8443-4d37fb1267f6" />

Root cause: The `DynamicPickleType` class in `database_session_service.py` configures MySQL dialect to use `LONGBLOB` for storing pickled data (line 117-118), but the `process_bind_param` and `process_result_value` methods only handle pickle serialization/deserialization for Spanner dialect, not MySQL. This causes MySQL to attempt storing raw Python objects instead of pickled bytes, leading to serialization errors and potential data corruption.

**Solution:**
Added MySQL to the pickle serialization logic in both `process_bind_param` and `process_result_value` methods, treating it the same way as Spanner dialect. This ensures that:
- Data is properly pickled to bytes before being stored in MySQL's LONGBLOB column
- Data is properly unpickled when retrieved from the database
- No breaking changes to existing functionality for other dialects (SQLite, PostgreSQL)

### Testing Plan

_Please describe the tests that you ran to verify your changes. This is required
for all PRs that are not small documentation or typo fixes._

**Unit Tests:**

- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.

**Summary of `pytest` results:**
<img width="929" height="306" alt="image" src="https://github.com/user-attachments/assets/3d548b96-ac49-4101-8405-a289a722293c" />

**Manual End-to-End (E2E) Tests:**

_Please provide instructions on how to manually test your changes, including any
necessary setup or configuration. Please provide logs or screenshots to help
reviewers better understand the fix._

### Checklist

- [x]  I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x]  I have performed a self-review of my own code.
- [x]  I have commented my code, particularly in hard-to-understand areas.
- [x]  I have added tests that prove my fix is effective or that my feature works.
- [x]  New and existing unit tests pass locally with my changes.
- [x]  I have manually tested my changes end-to-end.
- [x]  Any dependent changes have been merged and published in downstream modules.

### Additional context

_Add any other context or screenshots about the feature request here._

COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3282 from hung12ct:fix/mysql-pickle-serialization d9df37adb7dfbbfd8502a0fe65c4f8bca3d0d978
PiperOrigin-RevId: 825834360
2025-10-29 20:39:58 -07:00
Eliza Huang f9569bbb1a fix: Enable usage metadata in LiteLLM streaming
Closes #3181

Co-authored-by: Eliza Huang <heliza@google.com>
PiperOrigin-RevId: 825833660
2025-10-29 20:37:54 -07:00
Eliza Huang 71aa5645f6 fix: Propagate LiteLLM finish_reason to LlmResponse for use in callbacks
Closes #3114.

Co-authored-by: Eliza Huang <heliza@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3319 from lizzij:fix/litellm-finish-reason-3109 da6ed0aec514231fc274fee744f885ae95b9d45c
PiperOrigin-RevId: 825783229
2025-10-29 17:55:03 -07:00
Dongyu Jia 64294572c1 feat: Add get_job_info tool to BigQuery toolset
This CL introduces a new tool, get_job_info, to the BigQuery toolset. This tool allows retrieving metadata about a BigQuery job, such as slot usage, job configuration, statistics, and job status.

Closes #2928

Co-authored-by: Dongyu Jia <dongyuj@google.com>
PiperOrigin-RevId: 825762399
2025-10-29 16:46:31 -07:00
Ankur Sharma 72a8d8d85b fix: Session input file are required for creating an eval case
Session input file contains fields that are needed to run evals and later be able retrieve the session generated by them.

Co-authored-by: Ankur Sharma <ankusharma@google.com>
PiperOrigin-RevId: 825742522
2025-10-29 15:53:45 -07:00
Google Team Member 9014a849ea feat: Add api key argument to Vertex Session and Memory services for Express Mode support
We also change VertexAiSessionService and VertexAiMemoryBankService to both use keyword arguments for project, location, agent engine id, and express mode api key

PiperOrigin-RevId: 825719331
2025-10-29 14:54:12 -07:00
Vishwa Murugan c8e5340962 fix: Propagate MCP tools output schema to FunctionDeclaration
PiperOrigin-RevId: 825380145
2025-10-28 23:03:47 -07:00
Google Team Member 1ee93c8bcb fix: do not consider events with state delta and no content as final response
Previously this will return true for events yielded from before_agent_callback when there are state changes.

Note with this change, it will also return false for state delta only callbacks even after main response, but this is fine as long as the actual final response event has it to be true.

Closes #2992

PiperOrigin-RevId: 825313208
2025-10-28 19:36:34 -07:00
Kevin Qian 6bdac02dfc fix: do not consider events with state delta and no content as final response
Previously this will return true for events yielded from before_agent_callback when there are state changes.

Note with this change, it will also return false for state delta only callbacks even after main response, but this is fine as long as the actual final response event has it to be true.

Closes #2992

PiperOrigin-RevId: 825279439
2025-10-28 17:41:03 -07:00
Omar Elcircevi 74a3500fc5 fix: #3036 parameter filtering for CrewAI functions with **kwargs
Merge https://github.com/google/adk-python/pull/3037

fix: [#3036](https://github.com/google/adk-python/issues/3036)

- Fix FunctionTool parameter filtering to support CrewAI-style tools
- Functions with **kwargs now receive all parameters except 'self' and 'tool_context'
- Maintains backward compatibility with explicit parameter functions
- Add comprehensive tests for **kwargs functionality

Fixes parameter filtering issue where CrewAI tools using **kwargs pattern would receive empty parameter dictionaries, causing search_query and other parameters to be None.

#non-breaking

COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3037 from omarcevi:fix/function-tool-kwargs-parameter-filtering 012bbfcfd68e83a29635ac74718a1bd1323c5187
PiperOrigin-RevId: 825275686
2025-10-28 17:30:59 -07:00
Xinran (Sherry) Tang 420de9c5be test: Add a test case for resuming any invocation
PiperOrigin-RevId: 825266283
2025-10-28 17:03:07 -07:00
Ayush Agrawal 2f72ceb49b chore: Do not treat FinishReason.STOP as error case for LLM responses containing candidates with empty contents
Fixes #2905

PiperOrigin-RevId: 825263302
2025-10-28 16:54:50 -07:00
Wei Sun (Jack) 61adfcd69a refactor: Adds a util method to check the enablement of any env variable
PiperOrigin-RevId: 825199260
2025-10-28 14:18:45 -07:00
Google Team Member 9851340ad1 feat: Add Bigquery detect_anomalies tool
This change introduces a new `detect_anomalies` tool in `query_tool.py` which uses BigQuery ML's `CREATE MODEL` with `ARIMA_PLUS` type and `ML.DETECT_ANOMALIES` to detect anomalies. The new function is also added to the `bigquery_toolset`.

PiperOrigin-RevId: 825181489
2025-10-28 13:30:04 -07:00
George Weale 74d8361a7e fix: Add a fallback user message to LiteLLM requests if the last user message is empty
Related to #3255
Close #2560

PiperOrigin-RevId: 825143315
2025-10-28 11:56:47 -07:00
SOORAJ TS 240ef5beea feat: Added support for enums as arguments for function tools (#3088)
* feat: Added support for enums as arguments for function tools

* feat: Add default value support for function tools
fix: Add more test cases inside `test_build_function_declaration.py` for passing Enums as arguments

* fix: format code with pyink

---------

Co-authored-by: Wei Sun (Jack) <weisun@google.com>
Co-authored-by: Yvonne Yu <150068659+yyyu-google@users.noreply.github.com>
2025-10-28 10:35:02 -07:00
Ankur Sharma b17c8f19e5 chore: Marked expected_invocation as optional field on evaluator interface
ADK already has a set of metrics that don't rely expected_invocations. Also, for eval cases with conversation scenario, this would be the main line case.

PiperOrigin-RevId: 825101481
2025-10-28 10:27:47 -07:00
Shangjie Chen 9ab17f2afd fix: Yield the long running tool response before pausing execution
PiperOrigin-RevId: 825056377
2025-10-28 08:41:47 -07:00
Google Team Member 87dcb3f7ba feat: Add ApigeeLlm as a model that let's ADK Agent developers to connect with an Apigee proxy
PiperOrigin-RevId: 824712152
2025-10-27 15:53:33 -07:00
Ankur Sharma 00d147d62f fix: check if eval config file exists
PiperOrigin-RevId: 824709118
2025-10-27 15:45:14 -07:00