Commit Graph
67 Commits
Author SHA1 Message Date
Jaroslav PantsjohaandCopybara-Service 34d9b53f37 feat: Enhance error messages for tool and agent not found errors
Merge https://github.com/google/adk-python/pull/3219

## Summary

Enhance error messages for tool and agent not found errors to provide actionable guidance and reduce developer debugging time from hours to minutes.

Fixes #3217

## Changes

### Modified Files

1. **`src/google/adk/flows/llm_flows/functions.py`**
   - Enhanced `_get_tool()` error message with:
     - Available tools list (formatted, truncated to 20 for readability)
     - Possible causes
     - Suggested fixes
     - Fuzzy matching suggestions

2. **`src/google/adk/agents/llm_agent.py`**
   - Enhanced `__get_agent_to_run()` error message with:
     - Available agents list (formatted, truncated to 20 for readability)
     - Timing/ordering issue explanation
     - Fuzzy matching for agent names
   - Added `_get_available_agent_names()` helper method

### New Test Files

3. **`tests/unittests/flows/llm_flows/test_functions_error_messages.py`**
   - Tests for enhanced tool not found error messages
   - Fuzzy matching validation
   - Edge cases (no close matches, empty tools dict, 100+ tools)

4. **`tests/unittests/agents/test_llm_agent_error_messages.py`**
   - Tests for enhanced agent not found error messages
   - Agent tree traversal validation
   - Fuzzy matching for agents
   - Long list truncation

## Testing Plan

### Unit Tests

```bash
pytest tests/unittests/flows/llm_flows/test_functions_error_messages.py -v
pytest tests/unittests/agents/test_llm_agent_error_messages.py -v
```

**Results**:  8/8 tests passing

```
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_enhanced_error PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_fuzzy_matching PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_no_fuzzy_match PASSED
tests/unittests/flows/llm_flows/test_functions_error_messages.py::test_tool_not_found_truncates_long_list PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_enhanced_error PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_fuzzy_matching PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_tree_traversal PASSED
tests/unittests/agents/test_llm_agent_error_messages.py::test_agent_not_found_truncates_long_list PASSED

8 passed, 1 warning in 4.38s
```

### Example Enhanced Error Messages

#### Before (Current Error)

```
ValueError: Function get_equipment_specs is not found in the tools_dict: dict_keys(['get_equipment_details', 'query_vendor_catalog', 'score_proposals'])
```

#### After (Enhanced Error)

```
Function 'get_equipment_specs' is not found in available tools.

Available tools: get_equipment_details, query_vendor_catalog, score_proposals

Possible causes:
  1. LLM hallucinated the function name - review agent instruction clarity
  2. Tool not registered - verify agent.tools list
  3. Name mismatch - check for typos

Suggested fixes:
  - Review agent instruction to ensure tool usage is clear
  - Verify tool is included in agent.tools list
  - Check for typos in function name

Did you mean one of these?
  - get_equipment_details
```

## Community Impact

- **Addresses 3 active issues**: #2050, #2933 (12 comments), #2164
- **Reduces debugging time** from 3+ hours to < 5 minutes (validated in production multi-agent RFQ solution for recent partner nanothon initiative)
- **Improves developer experience** for new ADK users

## Implementation Details

- Uses standard library `difflib` for fuzzy matching (no new dependencies)
- Error path only (no performance impact on happy path)
- Measured performance: < 0.03ms per error
- Truncates long lists to first 20 items to prevent log overflow
- Fully backward compatible (same exception types)

## Checklist

- [x] Unit tests added and passing (8/8 tests)
- [x] Code formatted with `./autoformat.sh` (isort + pyink)
- [x] No new dependencies (uses standard library `difflib`)
- [x] Docstrings updated
- [x] Tested with Python 3.11
- [x] Issue #3217 created and linked

## Related Issues

- Fixes #3217
- Addresses #2050 - Tool verification callback request
- Addresses #2933 - How to handle "Function is not found in the tools_dict" Error
- Addresses #2164 - ValueError: {agent} not found in agent tree

---

**Note**: For production scenarios where LLM tool hallucinations occur, ADK's built-in [`ReflectAndRetryToolPlugin`](https://github.com/google/adk-python/blob/main/src/google/adk/plugins/reflect_retry_tool_plugin.py) can automatically retry failed tool calls (available since v1.16.0). This PR's enhanced error messages complement that by helping developers quickly identify and fix configuration issues during development.

Cheers, JP

Co-authored-by: Yvonne Yu <yyyu@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3219 from jpantsjoha:feat/better-error-messages a4df8bfb031685dce9e528d8eb7006f53447b75b
PiperOrigin-RevId: 826132579
2025-10-30 12:08:13 -07:00
George WealeandCopybara-Service 1e6a9daa63 fix: Change instruction insertion to respect tool call/response pairs
Make sure _add_instructions_to_user_content skips over user messages that carry function_response parts so tool_use/tool_result blocks stay together

Close #3229

Co-authored-by: George Weale <gweale@google.com>
PiperOrigin-RevId: 826076141
2025-10-30 09:54:49 -07:00
Ieva GrublyteandCopybara-Service c0892c725c chore: remove legacy validation that forbids co-exist of agent transfer and output_schema. Closes #3318
Remove validation for output_schema and agent transfer flags.

The check that prevented `output_schema` from co-existing with agent transfer capabilities (`disallow_transfer_to_parent` or `disallow_transfer_to_peers` being False) has been removed. The agent will no longer automatically set these transfer flags to True when `output_schema` is present.

Co-authored-by: Ieva Grublyte <ievagrublyte@google.com>
PiperOrigin-RevId: 825998224
2025-10-30 06:06:26 -07:00
Xuan YangandCopybara-Service 36ca4f1204 feat: Add on_tool_error_callback in LlmAgent
PiperOrigin-RevId: 824598502
2025-10-27 11:16:36 -07:00
Kathy WuandCopybara-Service d8c66fe688 fix: Update the saving artifact text for BuiltInCodeExecutor to be more human readable
Since it appears in the same bubble as the rest of the LLM's response text, make it more human readable so it doesn't look out of place.

PiperOrigin-RevId: 822729061
2025-10-22 13:57:41 -07:00
Wei Sun (Jack)andCopybara-Service 7d5c6b9acf fix: Fixes the identity prompt to be one line and add ending period after description statement
From

```
You are an agent. Your internal name is "agent".

 The description about you is "test description"
```

to

```
You are an agent. Your internal name is "agent". The description about you is "test description".
```

PiperOrigin-RevId: 822358196
2025-10-21 18:31:48 -07:00
Kathy WuandCopybara-Service ce3418a69d fix: Fix BuiltInCodeExecutor so that it can support visualizations
Previously BuiltInCodeExecutor was missing the logic to save output files from executed code as artifacts, so images/visualizations wouldn't show up in the UI. This fix will iterate through all parts of the LlmResponse, and if any of them are images, it will save the image data using artifact_service (similar to what is done in VertexAICodeExecutor).

This fixes the backend, but there are still UI bugs that should be fixed -- events without content are currently ignored, so the image doesn't appear even though it is saved. We will add the UI fix in a separate change.

PiperOrigin-RevId: 822245140
2025-10-21 13:06:43 -07:00
Shangjie ChenandCopybara-Service 9dce06f9b0 feat: Add rewind_async to support rewinding the session to before a previous invocation
PiperOrigin-RevId: 820552460
2025-10-16 23:24:40 -07:00
Xuan YangandCopybara-Service 3734ceaa6c fix: Use the agent's model when creating Google search agent tool
PiperOrigin-RevId: 819980797
2025-10-16 13:06:41 -07:00
Google Team MemberandCopybara-Service 6da7274858 feat: Set default for bypass_multi_tools_limit to False for GoogleSearchTool and VertexAiSearchTool
PiperOrigin-RevId: 818053371
2025-10-11 09:57:36 -07:00
Xiang (Sean) ZhouandCopybara-Service 9e0b1fb62b fix: Create context cache only when prefix matches with previous request
PiperOrigin-RevId: 817468275
2025-10-09 21:19:28 -07:00
Xiang (Sean) ZhouandCopybara-Service cc24d616f8 feat: Support ContentUnion as static instruction
PiperOrigin-RevId: 817278990
2025-10-09 11:52:24 -07:00
Xiang (Sean) ZhouandCopybara-Service c5b976b306 chore: Create the context cache based on the token count of previous request
before this change, we estimate the token count of the contents to cache and use it to compare with the threshold user set. but that's not precise , so we use the actual prompt token count of previous llm request.

We won't create cache for the very initial request

PiperOrigin-RevId: 814484840
2025-10-02 19:22:00 -07:00
Xuan YangandCopybara-Service d3148dacc9 ADK changes
PiperOrigin-RevId: 814319961
2025-10-02 13:44:05 -07:00
Shangjie ChenandCopybara-Service ce9c39f5a8 feat: Implement checkpoint and resume logic for LoopAgent
PiperOrigin-RevId: 813096880
2025-09-29 22:45:57 -07:00
Xinran (Sherry) TangandCopybara-Service f005414895 feat: Make resumable llm agents yield checkpoint events
PiperOrigin-RevId: 813001108
2025-09-29 17:08:58 -07:00
Xiang (Sean) ZhouandCopybara-Service ae5592e242 chore: Add tests for instruction provider and merge test_static_instructions.py to test_intructions.py
PiperOrigin-RevId: 810610507
2025-09-23 15:47:46 -07:00
Xiang (Sean) ZhouandCopybara-Service c66245a3b8 feat: support context caching
1. add a context cache config in app level which will apply to all agents in the app
2. pass on cache config through invocation context to llm_reqeust
3. store cache metadata in llm_response
4. lookup old cache metadata from latest event for reusing old cache
5. create new cache if old cache cannot be reused

PiperOrigin-RevId: 809158578
2025-09-19 13:17:02 -07:00
Afonso MenegolaandCopybara-Service c9ea80af28 fix: Prevent escaping of Latin characters in LLM response
Merge https://github.com/google/adk-python/pull/2937

**Closes #2936**

This Pull Request addresses the issue where `LlmAgent` outputs, when configured with `output_schema` and `tools`, were presenting escaped Latin characters (e.g., `\xf3` for `ó`) in the final response. This behavior occurred because `json.dumps` was being called with `ensure_ascii=True` (its default), which is not ideal for human-readable output, especially when dealing with non-ASCII characters common in many languages like Portuguese.

**Changes Proposed:**

* Modified the `_OutputSchemaRequestProcessor` in `src/google/adk/flows/llm_flows/_output_schema_processor.py` to explicitly set `ensure_ascii=False` when calling `json.dumps` for the `set_model_response` tool's output.

**Impact:**

This change ensures that all non-ASCII characters in the structured model response are preserved in their natural form, improving the readability and user experience of agent outputs, particularly for users interacting in languages with accented characters or other special symbols.

**Testing:**

The fix was verified locally by running an `LlmAgent` with an `output_schema` and confirming that responses containing Latin characters (e.g., "ação", "caminhão", "ícone") are now correctly displayed without escaping.

COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/2937 from amenegola:fix/issue-2936-escape-chars 6cac00f97aa4cd8d8ccaa97ec5fffc74f57995dc
PiperOrigin-RevId: 808622892
2025-09-18 09:48:03 -07:00
Kacper JawoszekandCopybara-Service 1ae0b82f56 feat(otel): add --otel_to_cloud experimental support
Cloud Trace, Cloud Monitoring and Cloud Logging integrations are set up via OTel if otel_to_cloud CLI param/fast_api arg is provided.

This is similar to current Cloud Trace integration via trace_to_cloud, just extended to Monitoring and Logging as well.

PiperOrigin-RevId: 807385680
2025-09-15 14:32:22 -07:00
Google Team MemberandCopybara-Service b1312680f4 feat(otel): add --otel_to_cloud experimental support
Cloud Trace, Cloud Monitoring and Cloud Logging integrations are set up via OTel if otel_to_cloud CLI param/fast_api arg is provided.

This is similar to current Cloud Trace integration via trace_to_cloud, just extended to Monitoring and Logging as well.

PiperOrigin-RevId: 807285744
2025-09-15 10:11:12 -07:00
Kacper JawoszekandCopybara-Service 7870480c63 feat(otel): add --otel_to_cloud experimental support
Cloud Trace, Cloud Monitoring and Cloud Logging integrations are set up via OTel if otel_to_cloud CLI param/fast_api arg is provided.

This is similar to current Cloud Trace integration via trace_to_cloud, just extended to Monitoring and Logging as well.

PiperOrigin-RevId: 807230668
2025-09-15 07:22:19 -07:00
Hangfei LinandCopybara-Service ab69ef8de8 feat: Move livebidi agents esp multi-agent to use session/events
The old live/bidi agents are using a cache to store context/history during agent transfer etc. As we have added support for session for live/bidi, we are now migrating the context/history cache to it. This improves scalability, efficiency and maintainability.

It introduces several changes:
* AudioTranscriber support is removed as now we are using native transcription from models.
* Transcription is returned as input_transcription/output_transcription fields and no longer as contents.
* We will return a new event with artifact references of file type of audio/pcm.(in addition to existing audio response event. So the users of this api need to do proper filtering here.)

PiperOrigin-RevId: 805997675
2025-09-11 14:58:33 -07:00
George WealeandCopybara-Service 43eec82f84 fix: Add a NOTE to agent transfer instructions listing available agents
The system instructions for agent transfer now include a NOTE section that lists all agents available for the `transfer_to_agent` function. This also has the target agents and, if there is one that applies, the parent agent. New unit tests are added to verify the correct generation of this NOTE.

PiperOrigin-RevId: 804569691
2025-09-08 13:43:38 -07:00
Wei Sun (Jack)andCopybara-Service a30851ee16 fix: Fixes thought handling in contents.py and refactors its unit tests
Before this change, other agent's reply with thought will still be inserted in the outgoing LlmRequest due to the wrong `else` statement for calling all other type of part.

This commit also refactors test_contents.py to be behavior-oriented tests, instead of implementation-oriented, and add more test cases to cover expected scenarios.

The tests are divided into the following files with different focus:

- test_contents.py: covers the basic logic of event filter;
- test_contents_branch.py: covers the behavior related to branch, which takes effect when ParallelAgent is used.
- test_contents_other_agent.py: covers the retelling behavior to include other agents' reply as context for the current agent.
- test_contents_function.py: covers the function_call/function_response rearrangement logic mainly for `LongRunningFunctionTool`.

PiperOrigin-RevId: 802759821
2025-09-03 16:42:20 -07:00