Compare commits

..

8 Commits

Author SHA1 Message Date
Google Team Member 068df04bce fix: import deprecated from typing_extensions
Fixes https://github.com/google/adk-python/issues/1125

and add it to pyproject.toml. This was added to typing_extensions in 4.5.0. I followed versioning guidance from https://typing-extensions.readthedocs.io/en/latest/index.html#versioning-and-backwards-compatibility.

PiperOrigin-RevId: 767317467
2025-06-04 15:51:43 -07:00
Google Team Member 86e15cab89 chore: Make ArtifactService transparent to AgentTools
PiperOrigin-RevId: 767225493
2025-06-04 11:35:49 -07:00
Selcuk Gun 92e7a4a488 chore: bump version number to 1.2.0
PiperOrigin-RevId: 767218068
2025-06-04 11:18:30 -07:00
Shangjie Chen c6e1e82efb chore: Minor improvement to session service
- Add missing override.
- Add warning to failed actions.
- Remove unused import.
- Remove unused fields.
- Add type checking.

PiperOrigin-RevId: 767209697
2025-06-04 10:58:35 -07:00
Google Team Member 54ed031d1a feat: support None as return type, such as def func() -> None:
None:

PiperOrigin-RevId: 767204150
2025-06-04 10:43:35 -07:00
Ankur Sharma 984c1d6b8b test: Added unit test coverage for local_eval_set_results_manager.py
PiperOrigin-RevId: 767202237
2025-06-04 10:39:09 -07:00
Copybara-Service 552e42fa06 Merge pull request #1154 from google:hangfei-patch-2
PiperOrigin-RevId: 767197024
2025-06-04 10:25:07 -07:00
Hangfei Lin d052e8f13a chore: Update agent.py 2025-06-04 09:29:33 -07:00
14 changed files with 453 additions and 55 deletions
+84
View File
@@ -1,5 +1,89 @@
# Changelog
## [1.2.0](https://github.com/google/adk-python/compare/v1.1.1...v1.2.0) (2025-06-04)
### Features
* Add agent engine as a deployment option to the ADK CLI ([2409c3e](https://github.com/google/adk-python/commit/2409c3ef192262c80f5328121f6dc4f34265f5cf))
* Add an option to use gcs artifact service in adk web. ([8d36dbd](https://github.com/google/adk-python/commit/8d36dbda520b1c0dec148e1e1d84e36ddcb9cb95))
* Add index tracking to handle parallel tool call using litellm ([05f4834](https://github.com/google/adk-python/commit/05f4834759c9b1f0c0af9d89adb7b81ea67d82c8))
* Add sortByColumn functionality to List Operation ([af95dd2](https://github.com/google/adk-python/commit/af95dd29325865ec30a1945b98e65e457760e003))
* Add implementation for `get_eval_case`, `update_eval_case` and `delete_eval_case` for the local eval sets manager. ([a7575e0](https://github.com/google/adk-python/commit/a7575e078a564af6db3f42f650e94ebc4f338918))
* Expose more config of VertexAiSearchTool from latest Google GenAI SDK ([2b5c89b](https://github.com/google/adk-python/commit/2b5c89b3a94e82ea4a40363ea8de33d9473d7cf0))
* New Agent Visualization ([da4bc0e](https://github.com/google/adk-python/commit/da4bc0efc0dd96096724559008205854e97c3fd1))
* Set the max width and height of view image dialog to be 90% ([98a635a](https://github.com/google/adk-python/commit/98a635afee399f64e0a813d681cd8521fbb49500))
* Support Langchain StructuredTool for Langchain tool ([7e637d3](https://github.com/google/adk-python/commit/7e637d3fa05ca3e43a937e7158008d2b146b1b81))
* Support Langchain tools that has run_manager in _run args and don't have args_schema populated ([3616bb5](https://github.com/google/adk-python/commit/3616bb5fc4da90e79eb89039fb5e302d6a0a14ec))
* Update for anthropic models ([16f7d98](https://github.com/google/adk-python/commit/16f7d98acf039f21ec8a99f19eabf0ef4cb5268c))
* Use bigquery scope by default in bigquery credentials. ([ba5b80d](https://github.com/google/adk-python/commit/ba5b80d5d774ff5fdb61bd43b7849057da2b4edf))
* Add jira_agent adk samples code which connect Jira cloud ([8759a25](https://github.com/google/adk-python/commit/8759a2525170edb2f4be44236fa646a93ba863e6))
* Render HTML artifact in chat window ([5c2ad32](https://github.com/google/adk-python/commit/5c2ad327bf4262257c3bc91010c3f8c303d3a5f5))
* Add export to json button in the chat window ([fc3e374](https://github.com/google/adk-python/commit/fc3e374c86c4de87b4935ee9c56b6259f00e8ea2))
* Add tooltip to the export session button ([2735942](https://github.com/google/adk-python/commit/273594215efe9dbed44d4ef85e6234bd7ba7b7ae))
### Bug Fixes
* Add adk icon for UI ([2623c71](https://github.com/google/adk-python/commit/2623c710868d832b6d5119f38e22d82adb3de66b))
* Add cache_ok option to remove sa warning. ([841e10a](https://github.com/google/adk-python/commit/841e10ae353e0b1b3d020a26d6cac6f37981550e))
* Add support for running python main function in UnsafeLocalCodeExecutor when the code has an if __name__ == "__main__" statement. ([95e33ba](https://github.com/google/adk-python/commit/95e33baf57e9c267a758e08108cde76adf8af69b))
* Adk web not working on some env for windows, fixes https://github.com/google/adk-web/issues/34 ([daac8ce](https://github.com/google/adk-python/commit/daac8cedfe6d894f77ea52784f0a6d19003b2c00))
* Assign empty inputSchema to MCP tool when converting an ADK tool that wraps a function which takes no parameters. ([2a65c41](https://github.com/google/adk-python/commit/2a65c4118bb2aa97f2a13064db884bd63c14a5f7))
* Call all tools in parallel calls during partial authentication ([0e72efb](https://github.com/google/adk-python/commit/0e72efb4398ce6a5d782bcdcb770b2473eb5af2e))
* Continue fetching events if there are multiple pages. ([6506302](https://github.com/google/adk-python/commit/65063023a5a7cb6cd5db43db14a411213dc8acf5))
* Do not convert "false" value to dict ([60ceea7](https://github.com/google/adk-python/commit/60ceea72bde2143eb102c60cf33b365e1ab07d8f))
* Enhance agent loader exception handler and expose precise error information ([7b51ae9](https://github.com/google/adk-python/commit/7b51ae97245f6990c089183734aad41fe59b3330))
* Ensure function description is copied when ignoring parameters ([7fdc6b4](https://github.com/google/adk-python/commit/7fdc6b4417e5cf0fbc72d3117531914353d3984a))
* Filter memory by app_name and user_id. ([db4bc98](https://github.com/google/adk-python/commit/db4bc9809c7bb6b0d261973ca7cfd87b392694be))
* Fix filtering by user_id for vertex ai session service listing ([9d4ca4e](https://github.com/google/adk-python/commit/9d4ca4ed44cf10bc87f577873faa49af469acc25))
* fix parameter schema generation for gemini ([5a67a94](https://github.com/google/adk-python/commit/5a67a946d2168b80dd6eba008218468c2db2e74e))
* Handle non-indexed function call chunks with incremental fallback index ([b181cbc](https://github.com/google/adk-python/commit/b181cbc8bc629d1c9bfd50054e47a0a1b04f7410))
* Handles function tool parsing corner case where type hints are stored as strings. ([a8a2074](https://github.com/google/adk-python/commit/a8a20743f92cd63c3d287a3d503c1913dd5ad5ae))
* Introduce PreciseTimestamp to fix mysql datetime precision issue. ([841e10a](https://github.com/google/adk-python/commit/841e10ae353e0b1b3d020a26d6cac6f37981550e))
* match arg case in errors ([b226a06](https://github.com/google/adk-python/commit/b226a06c0bf798f85a53c591ad12ee582703af6d))
* ParallelAgent should only append to its immediate sub-agent, not transitive descendants ([ec8bc73](https://github.com/google/adk-python/commit/ec8bc7387c84c3f261c44cedfe76eb1f702e7b17))
* Relax openapi spec to gemini schema conversion to tolerate more cases ([b1a74d0](https://github.com/google/adk-python/commit/b1a74d099fae44d41750b79e58455282d919dd78))
* Remove labels from config when using API key from Google AI Studio to call model ([5d29716](https://github.com/google/adk-python/commit/5d297169d08a2d0ea1a07641da2ac39fa46b68a4))
* **sample:** Correct text artifact saving in artifact_save_text sample ([5c6001d](https://github.com/google/adk-python/commit/5c6001d90fe6e1d15a2db6b30ecf9e7b6c26eee4))
* Separate thinking from text parts in streaming mode ([795605a](https://github.com/google/adk-python/commit/795605a37e1141e37d86c9b3fa484a3a03e7e9a6))
* Simplify content for ollama provider ([eaee49b](https://github.com/google/adk-python/commit/eaee49bc897c20231ecacde6855cccfa5e80d849))
* Timeout issues for mcpstdio server when mcp tools are incorrect. ([45ef668](https://github.com/google/adk-python/commit/45ef6684352e3c8082958bece8610df60048f4a3))
* **transfer_to_agent:** update docstring for clarity and accuracy ([854a544](https://github.com/google/adk-python/commit/854a5440614590c2a3466cf652688ba57d637205))
* Update unit test code for test_connection ([b0403b2](https://github.com/google/adk-python/commit/b0403b2d98b2776d15475f6b525409670e2841fc))
* Use inspect.cleandoc on function docstrings in generate_function_declaration. ([f7cb666](https://github.com/google/adk-python/commit/f7cb66620be843b8d9f3d197d6e8988e9ee0dfca))
* Restore errors path ([32c5ffa](https://github.com/google/adk-python/commit/32c5ffa8ca5e037f41ff345f9eecf5b26f926ea1))
* Unused import for deprecated ([ccd05e0](https://github.com/google/adk-python/commit/ccd05e0b00d0327186e3b1156f1b0216293efe21))
* Prevent JSON parsing errors and preserve non-ascii characters in telemetry ([d587270](https://github.com/google/adk-python/commit/d587270327a8de9f33b3268de5811ac756959850))
* Raise HTTPException when running evals in fast_api if google-adk[eval] is not installed ([1de5c34](https://github.com/google/adk-python/commit/1de5c340d8da1cedee223f6f5a8c90070a9f0298))
* Fix typos in README for sample bigquery_agent and oauth_calendar_agent ([9bdd813](https://github.com/google/adk-python/commit/9bdd813be15935af5c5d2a6982a2391a640cab23))
* Make tool_call one span for telemetry and renamed to execute_tool ([999a7fe](https://github.com/google/adk-python/commit/999a7fe69d511b1401b295d23ab3c2f40bccdc6f))
* Use media type in chat window. Remove isArtifactImage and isArtifactAudio reference ([1452dac](https://github.com/google/adk-python/commit/1452dacfeb6b9970284e1ddeee6c4f3cb56781f8))
* Set output_schema correctly for LiteLllm ([6157db7](https://github.com/google/adk-python/commit/6157db77f2fba4a44d075b51c83bff844027a147))
* Update pending event dialog style ([1db601c](https://github.com/google/adk-python/commit/1db601c4bd90467b97a2f26fe9d90d665eb3c740))
* Remove the gap between event holder and image ([63822c3](https://github.com/google/adk-python/commit/63822c3fa8b0bdce2527bd0d909c038e2b66dd98))
### Documentation
* Adds a sample agent to illustrate state usage via `callbacks`. ([18fbe3c](https://github.com/google/adk-python/commit/18fbe3cbfc9f2af97e4b744ec0a7552331b1d8e3))
* Fix typos in documentation ([7aaf811](https://github.com/google/adk-python/commit/7aaf8116169c210ceda35c649b5b49fb65bbb740))
* Change eval_dataset to eval_dataset_file_path_or_dir ([62d7bf5](https://github.com/google/adk-python/commit/62d7bf58bb1c874caaf3c56a614500ae3b52f215))
* Fix broken link to A2A example ([0d66a78](https://github.com/google/adk-python/commit/0d66a7888b68380241b92f7de394a06df5a0cc06))
* Fix typo in envs.py ([bd588bc](https://github.com/google/adk-python/commit/bd588bce50ccd0e70b96c7291db035a327ad4d24))
* Updates CONTRIBUTING.md to refine setup process using uv. ([04e07b4](https://github.com/google/adk-python/commit/04e07b4a1451123272641a256c6af1528ea6523e))
* Create and update project documentation including README.md and CONTRIBUTING.md ([f180331](https://github.com/google/adk-python/commit/f1803312c6a046f94c23cfeaed3e8656afccf7c3))
* Rename the root agent in the example to match the example name ([94c0aca](https://github.com/google/adk-python/commit/94c0aca685f1dfa4edb44caaedc2de25cc0caa41))
* ADK: add section comment ([349a414](https://github.com/google/adk-python/commit/349a414120fbff0937966af95864bd683f063d08))
### Chore
* Miscellaneous changes ([0724a83](https://github.com/google/adk-python/commit/0724a83aa9cda00c1b228ed47a5baa7527bb4a0a), [a9dcc58](https://github.com/google/adk-python/commit/a9dcc588ad63013d063dbe37095c0d2e870142c3), [ac52eab](https://github.com/google/adk-python/commit/ac52eab88eccafa451be7584e24aea93ff15f3f3), [a0714b8](https://github.com/google/adk-python/commit/a0714b8afc55461f315ede8451b17aad18d698dd))
* Enable release-please workflow ([57d99aa](https://github.com/google/adk-python/commit/57d99aa7897fb229f41c2a08034606df1e1e6064))
* Added unit test coverage for local_eval_sets_manager.py ([174afb3](https://github.com/google/adk-python/commit/174afb3975bdc7e5f10c26f3eebb17d2efa0dd59))
* Extract common options for `adk web` and `adk api_server` ([01965bd](https://github.com/google/adk-python/commit/01965bdd74a9dbdb0ce91a924db8dee5961478b8))
## 1.1.1
### Features
@@ -61,7 +61,7 @@ async def check_prime(nums: list[int]) -> str:
root_agent = Agent(
model=Claude(model="claude-3-5-sonnet-v2@20241022"),
name="data_processing_agent",
name="hello_world_agent",
description=(
"hello world agent that can roll a dice of 8 sides and check prime"
" numbers."
+1
View File
@@ -44,6 +44,7 @@ dependencies = [
"PyYAML>=6.0.2", # For APIHubToolset.
"sqlalchemy>=2.0", # SQL database ORM
"tzlocal>=5.3", # Time zone utilities
"typing-extensions>=4.5, <5",
"uvicorn>=0.34.0", # ASGI server for FastAPI
# go/keep-sorted end
]
+4 -2
View File
@@ -12,18 +12,20 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
from typing import Any
from typing import Tuple
from deprecated import deprecated
from google.genai import types as genai_types
from typing_extensions import deprecated
from ...evaluation.eval_case import IntermediateData
from ...evaluation.eval_case import Invocation
from ...sessions.session import Session
@deprecated(reason='Use convert_session_to_eval_invocations instead.')
@deprecated('Use convert_session_to_eval_invocations instead.')
def convert_session_to_eval_format(session: Session) -> list[dict[str, Any]]:
"""Converts a session data into eval format.
@@ -12,13 +12,15 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
from typing import Any
from typing import Optional
from deprecated import deprecated
from google.genai import types as genai_types
import pandas as pd
from tabulate import tabulate
from typing_extensions import deprecated
from typing_extensions import override
from vertexai.preview.evaluation import EvalTask
from vertexai.preview.evaluation import MetricPromptTemplateExamples
@@ -124,10 +126,8 @@ class ResponseEvaluator(Evaluator):
@staticmethod
@deprecated(
reason=(
"This method has been deprecated and will be removed soon. Please use"
" evaluate_invocations instead."
)
"This method has been deprecated and will be removed soon. Please use"
" evaluate_invocations instead."
)
def evaluate(
raw_eval_dataset: list[list[dict[str, Any]]],
@@ -12,13 +12,15 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
from typing import Any
from typing import cast
from deprecated import deprecated
from google.genai import types as genai_types
import pandas as pd
from tabulate import tabulate
from typing_extensions import deprecated
from typing_extensions import override
from .eval_case import Invocation
@@ -100,10 +102,8 @@ class TrajectoryEvaluator(Evaluator):
@staticmethod
@deprecated(
reason=(
"This method has been deprecated and will be removed soon. Please use"
" evaluate_invocations instead."
)
"This method has been deprecated and will be removed soon. Please use"
" evaluate_invocations instead."
)
def evaluate(
eval_dataset: list[list[dict[str, Any]]],
@@ -218,7 +218,10 @@ class TrajectoryEvaluator(Evaluator):
return new_row, failure
@staticmethod
@deprecated()
@deprecated(
"are_tools_equal is deprecated and will be removed soon. Please use"
" TrajectoryEvaluator._are_tool_calls_equal instead."
)
def are_tools_equal(list_a_original, list_b_original):
# Remove other entries that we don't want to evaluate
list_a = [
@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
import copy
import logging
@@ -223,6 +224,7 @@ class InMemorySessionService(BaseSessionService):
sessions_without_events.append(copied_session)
return ListSessionsResponse(sessions=sessions_without_events)
@override
async def delete_session(
self, *, app_name: str, user_id: str, session_id: str
) -> None:
@@ -247,7 +249,7 @@ class InMemorySessionService(BaseSessionService):
)
is None
):
return None
return
self.sessions[app_name][user_id].pop(session_id)
@@ -261,11 +263,20 @@ class InMemorySessionService(BaseSessionService):
app_name = session.app_name
user_id = session.user_id
session_id = session.id
def _warning(message: str) -> None:
logger.warning(
f'Failed to append event to session {session_id}: {message}'
)
if app_name not in self.sessions:
_warning(f'app_name {app_name} not in sessions')
return event
if user_id not in self.sessions[app_name]:
_warning(f'user_id {user_id} not in sessions[app_name]')
return event
if session_id not in self.sessions[app_name][user_id]:
_warning(f'session_id {session_id} not in sessions[app_name][user_id]')
return event
if event.actions and event.actions.state_delta:
@@ -16,8 +16,8 @@ from __future__ import annotations
import asyncio
import logging
import re
import time
from typing import Any
from typing import Dict
from typing import Optional
import urllib.parse
@@ -50,9 +50,6 @@ class VertexAiSessionService(BaseSessionService):
self.project = project
self.location = location
client = genai.Client(vertexai=True, project=project, location=location)
self.api_client = client._api_client
@override
async def create_session(
self,
@@ -86,6 +83,7 @@ class VertexAiSessionService(BaseSessionService):
operation_id = api_response['name'].split('/')[-1]
max_retry_attempt = 5
lro_response = None
while max_retry_attempt >= 0:
lro_response = await api_client.async_request(
http_method='GET',
@@ -99,6 +97,11 @@ class VertexAiSessionService(BaseSessionService):
await asyncio.sleep(1)
max_retry_attempt -= 1
if lro_response is None or not lro_response.get('done', None):
raise TimeoutError(
f'Timeout waiting for operation {operation_id} to complete.'
)
# Get session resource
get_session_api_response = await api_client.async_request(
http_method='GET',
@@ -235,11 +238,15 @@ class VertexAiSessionService(BaseSessionService):
) -> None:
reasoning_engine_id = _parse_reasoning_engine_id(app_name)
api_client = _get_api_client(self.project, self.location)
await api_client.async_request(
http_method='DELETE',
path=f'reasoningEngines/{reasoning_engine_id}/sessions/{session_id}',
request_dict={},
)
try:
await api_client.async_request(
http_method='DELETE',
path=f'reasoningEngines/{reasoning_engine_id}/sessions/{session_id}',
request_dict={},
)
except Exception as e:
logger.error(f'Error deleting session {session_id}: {e}')
raise e
@override
async def append_event(self, session: Session, event: Event) -> Event:
@@ -266,7 +273,7 @@ def _get_api_client(project: str, location: str):
return client._api_client
def _convert_event_to_json(event: Event):
def _convert_event_to_json(event: Event) -> Dict[str, Any]:
metadata_json = {
'partial': event.partial,
'turn_complete': event.turn_complete,
@@ -318,7 +325,7 @@ def _convert_event_to_json(event: Event):
return event_json
def _from_api_event(api_event: dict) -> Event:
def _from_api_event(api_event: Dict[str, Any]) -> Event:
event_actions = EventActions()
if api_event.get('actions', None):
event_actions = EventActions(
@@ -0,0 +1,96 @@
# Copyright 2025 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
from typing import Optional
from typing import TYPE_CHECKING
from google.genai import types
from typing_extensions import override
from ..artifacts.base_artifact_service import BaseArtifactService
if TYPE_CHECKING:
from .tool_context import ToolContext
class ForwardingArtifactService(BaseArtifactService):
"""Artifact service that forwards to the parent tool context."""
def __init__(self, tool_context: ToolContext):
self.tool_context = tool_context
self._invocation_context = tool_context._invocation_context
@override
async def save_artifact(
self,
*,
app_name: str,
user_id: str,
session_id: str,
filename: str,
artifact: types.Part,
) -> int:
return await self.tool_context.save_artifact(
filename=filename, artifact=artifact
)
@override
async def load_artifact(
self,
*,
app_name: str,
user_id: str,
session_id: str,
filename: str,
version: Optional[int] = None,
) -> Optional[types.Part]:
return await self.tool_context.load_artifact(
filename=filename, version=version
)
@override
async def list_artifact_keys(
self, *, app_name: str, user_id: str, session_id: str
) -> list[str]:
return await self.tool_context.list_artifacts()
@override
async def delete_artifact(
self, *, app_name: str, user_id: str, session_id: str, filename: str
) -> None:
del app_name, user_id, session_id
if self._invocation_context.artifact_service is None:
raise ValueError("Artifact service is not initialized.")
await self._invocation_context.artifact_service.delete_artifact(
app_name=self._invocation_context.app_name,
user_id=self._invocation_context.user_id,
session_id=self._invocation_context.session.id,
filename=filename,
)
@override
async def list_versions(
self, *, app_name: str, user_id: str, session_id: str, filename: str
) -> list[int]:
del app_name, user_id, session_id
if self._invocation_context.artifact_service is None:
raise ValueError("Artifact service is not initialized.")
return await self._invocation_context.artifact_service.list_versions(
app_name=self._invocation_context.app_name,
user_id=self._invocation_context.user_id,
session_id=self._invocation_context.session.id,
filename=filename,
)
@@ -37,6 +37,7 @@ _py_builtin_type_to_schema_type = {
bool: types.Type.BOOLEAN,
list: types.Type.ARRAY,
dict: types.Type.OBJECT,
None: types.Type.NULL,
}
logger = logging.getLogger('google_adk.' + __name__)
+2 -21
View File
@@ -25,6 +25,7 @@ from . import _automatic_function_calling_util
from ..memory.in_memory_memory_service import InMemoryMemoryService
from ..runners import Runner
from ..sessions.in_memory_session_service import InMemorySessionService
from ._forwarding_artifact_service import ForwardingArtifactService
from .base_tool import BaseTool
from .tool_context import ToolContext
@@ -123,9 +124,7 @@ class AgentTool(BaseTool):
runner = Runner(
app_name=self.agent.name,
agent=self.agent,
# TODO(kech): Remove the access to the invocation context.
# It seems we don't need re-use artifact_service if we forward below.
artifact_service=tool_context._invocation_context.artifact_service,
artifact_service=ForwardingArtifactService(tool_context),
session_service=InMemorySessionService(),
memory_service=InMemoryMemoryService(),
)
@@ -144,24 +143,6 @@ class AgentTool(BaseTool):
tool_context.state.update(event.actions.state_delta)
last_event = event
if runner.artifact_service:
# Forward all artifacts to parent session.
artifact_names = await runner.artifact_service.list_artifact_keys(
app_name=session.app_name,
user_id=session.user_id,
session_id=session.id,
)
for artifact_name in artifact_names:
if artifact := await runner.artifact_service.load_artifact(
app_name=session.app_name,
user_id=session.user_id,
session_id=session.id,
filename=artifact_name,
):
await tool_context.save_artifact(
filename=artifact_name, artifact=artifact
)
if not last_event or not last_event.content or not last_event.content.parts:
return ''
if isinstance(self.agent, LlmAgent) and self.agent.output_schema:
+2 -2
View File
@@ -12,5 +12,5 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# version: date+base_cl
__version__ = "1.1.1"
# version: major.minor.patch
__version__ = "1.2.0"
@@ -0,0 +1,168 @@
# Copyright 2025 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import annotations
import json
import os
import shutil
import tempfile
import time
from unittest.mock import patch
from google.adk.evaluation.eval_result import EvalCaseResult
from google.adk.evaluation.eval_result import EvalSetResult
from google.adk.evaluation.evaluator import EvalStatus
from google.adk.evaluation.local_eval_set_results_manager import _ADK_EVAL_HISTORY_DIR
from google.adk.evaluation.local_eval_set_results_manager import _EVAL_SET_RESULT_FILE_EXTENSION
from google.adk.evaluation.local_eval_set_results_manager import _sanitize_eval_set_result_name
from google.adk.evaluation.local_eval_set_results_manager import LocalEvalSetResultsManager
import pytest
def test_sanitize_eval_set_result_name():
assert _sanitize_eval_set_result_name("app/name") == "app_name"
assert _sanitize_eval_set_result_name("app_name") == "app_name"
assert _sanitize_eval_set_result_name("app/name/with/slashes") == (
"app_name_with_slashes"
)
class TestLocalEvalSetResultsManager:
@pytest.fixture(autouse=True)
def setup(self):
self.temp_dir = tempfile.mkdtemp()
self.agents_dir = os.path.join(self.temp_dir, "agents")
os.makedirs(self.agents_dir)
self.manager = LocalEvalSetResultsManager(self.agents_dir)
self.app_name = "test_app"
self.eval_set_id = "test_eval_set"
self.eval_case_results = [
EvalCaseResult(
eval_set_file="test_file",
eval_set_id=self.eval_set_id,
eval_id="case1",
final_eval_status=EvalStatus.PASSED,
eval_metric_results=[],
overall_eval_metric_results=[],
eval_metric_result_per_invocation=[],
session_id="session1",
)
]
self.timestamp = time.time() # Store the timestamp
self.eval_set_result_id = (
self.app_name + "_" + self.eval_set_id + "_" + str(self.timestamp)
)
self.eval_set_result_name = _sanitize_eval_set_result_name(
self.eval_set_result_id
)
self.eval_set_result = EvalSetResult(
eval_set_result_id=self.eval_set_result_id,
eval_set_result_name=self.eval_set_result_name,
eval_set_id=self.eval_set_id,
eval_case_results=self.eval_case_results,
creation_timestamp=self.timestamp,
)
def teardown(self):
shutil.rmtree(self.temp_dir)
@patch("time.time")
def test_save_eval_set_result(self, mock_time):
mock_time.return_value = self.timestamp
self.manager.save_eval_set_result(
self.app_name, self.eval_set_id, self.eval_case_results
)
eval_history_dir = os.path.join(
self.agents_dir, self.app_name, _ADK_EVAL_HISTORY_DIR
)
expected_file_path = os.path.join(
eval_history_dir,
self.eval_set_result_name + _EVAL_SET_RESULT_FILE_EXTENSION,
)
assert os.path.exists(expected_file_path)
with open(expected_file_path, "r") as f:
actual_eval_set_result_json = json.load(f)
# need to convert eval_set_result to json
expected_eval_set_result_json = self.eval_set_result.model_dump_json()
assert expected_eval_set_result_json == actual_eval_set_result_json
@patch("time.time")
def test_get_eval_set_result(self, mock_time):
mock_time.return_value = self.timestamp
self.manager.save_eval_set_result(
self.app_name, self.eval_set_id, self.eval_case_results
)
retrieved_result = self.manager.get_eval_set_result(
self.app_name, self.eval_set_result_name
)
assert retrieved_result == self.eval_set_result
@patch("time.time")
def test_get_eval_set_result_not_found(self, mock_time):
mock_time.return_value = self.timestamp
with pytest.raises(ValueError) as e:
self.manager.get_eval_set_result(self.app_name, "non_existent_id")
assert "does not exist" in str(e.value)
@patch("time.time")
def test_list_eval_set_results(self, mock_time):
mock_time.return_value = self.timestamp
# Save two eval set results for the same app
self.manager.save_eval_set_result(
self.app_name, self.eval_set_id, self.eval_case_results
)
timestamp2 = time.time() + 1
mock_time.return_value = timestamp2
eval_set_result_id2 = (
self.app_name + "_" + self.eval_set_id + "_" + str(timestamp2)
)
eval_set_result_name2 = _sanitize_eval_set_result_name(eval_set_result_id2)
eval_case_results2 = [
EvalCaseResult(
eval_set_file="test_file",
eval_set_id=self.eval_set_id,
eval_id="case2",
final_eval_status=EvalStatus.FAILED,
eval_metric_results=[],
overall_eval_metric_results=[],
eval_metric_result_per_invocation=[],
session_id="session2",
)
]
self.manager.save_eval_set_result(
self.app_name, self.eval_set_id, eval_case_results2
)
# Save one eval set result for a different app
app_name2 = "another_app"
timestamp3 = time.time() + 2
mock_time.return_value = timestamp3
self.manager.save_eval_set_result(
app_name2, self.eval_set_id, self.eval_case_results
)
results = self.manager.list_eval_set_results(self.app_name)
expected_result = [self.eval_set_result_name, eval_set_result_name2]
assert set(results) == set(expected_result)
def test_list_eval_set_results_empty(self):
# No eval set results saved for the app
results = self.manager.list_eval_set_results(self.app_name)
assert results == []
+50 -6
View File
@@ -13,20 +13,15 @@
# limitations under the License.
from google.adk.agents import Agent
from google.adk.agents import SequentialAgent
from google.adk.agents.callback_context import CallbackContext
from google.adk.tools.agent_tool import AgentTool
from google.genai.types import Part
from pydantic import BaseModel
import pytest
from pytest import mark
from .. import testing_utils
pytestmark = pytest.mark.skip(
reason='Skipping until tool.func evaluations are fixed (async)'
)
function_call_custom = Part.from_function_call(
name='tool_agent', args={'custom_input': 'test1'}
)
@@ -112,6 +107,55 @@ def test_update_state():
assert runner.session.state['state_1'] == 'changed_value'
def test_update_artifacts():
"""The agent tool can read and write artifacts."""
async def before_tool_agent(callback_context: CallbackContext):
# Artifact 1 should be available in the tool agent.
artifact = await callback_context.load_artifact('artifact_1')
await callback_context.save_artifact(
'artifact_2', Part.from_text(text=artifact.text + ' 2')
)
tool_agent = SequentialAgent(
name='tool_agent',
before_agent_callback=before_tool_agent,
)
async def before_main_agent(callback_context: CallbackContext):
await callback_context.save_artifact(
'artifact_1', Part.from_text(text='test')
)
async def after_main_agent(callback_context: CallbackContext):
# Artifact 2 should be available after the tool agent.
artifact_2 = await callback_context.load_artifact('artifact_2')
await callback_context.save_artifact(
'artifact_3', Part.from_text(text=artifact_2.text + ' 3')
)
mock_model = testing_utils.MockModel.create(
responses=[function_call_no_schema, 'response2']
)
root_agent = Agent(
name='root_agent',
before_agent_callback=before_main_agent,
after_agent_callback=after_main_agent,
tools=[AgentTool(agent=tool_agent)],
model=mock_model,
)
runner = testing_utils.InMemoryRunner(root_agent)
runner.run('test1')
artifacts_path = f'test_app/test_user/{runner.session_id}'
assert runner.runner.artifact_service.artifacts == {
f'{artifacts_path}/artifact_1': [Part.from_text(text='test')],
f'{artifacts_path}/artifact_2': [Part.from_text(text='test 2')],
f'{artifacts_path}/artifact_3': [Part.from_text(text='test 2 3')],
}
@mark.parametrize(
'env_variables',
[