The evaluate_invocations method override in Evaluator subclasses was not consistent, leading to errors during calls, especially when using kwargs. Made the overrides and calls consistent to resolve this issue.
Co-authored-by: Keyur Joshi <keyurj@google.com>
PiperOrigin-RevId: 850462752
This is about 35% decrease. This change refactors several ADK modules to import `vertexai` and its submodules only when they are first used, rather than at the top of the file. This improves module load times by avoiding unnecessary imports of large dependencies. Imports are also placed within `if TYPE_CHECKING:` blocks where appropriate.
Co-authored-by: Liang Wu <wuliang@google.com>
PiperOrigin-RevId: 829017293
ADK already has a set of metrics that don't rely expected_invocations. Also, for eval cases with conversation scenario, this would be the main line case.
PiperOrigin-RevId: 825101481
This endpoint could be used by ADK Web to dynamically know:
- What are the available eval metrics in an App
- A description of those metrics
- A value range supported by those metrics
We also update the metric registry to make it mandatory to supply these details. The goal is to improve usability and interpretability of the eval metrics.
PiperOrigin-RevId: 787277695
We add a new metric for evaluating safety of Agent's response to ADK Eval. We delegate the actual implementation to Vertex Gen AI Eval SDK, so using this metric will require GCP project.
As a part of this change, we created (refactored) a simple Facade for vertex gen ai eval sdk.
PiperOrigin-RevId: 778580406
--
ad923c2c8c503ba73c62db695e88f1a3ea1aeeea by YU MING HSU <abego452@gmail.com>:
docs: enhance Contribution process within CONTRIBUTING.md
--
8022924fb7e975ac278d38fce3b5fd593d874536 by YU MING HSU <abego452@gmail.com>:
fix: move _maybe_append_user_content from google_llm.py to base_llm.py,
so subclass can get benefit from it, call _maybe_append_user_content
from generate_content_async within lite_llm.py
--
cf891fb1a3bbccaaf9d0055b23f614ce52449977 by YU MING HSU <abego452@gmail.com>:
fix: modify install dependencies cmd, and use pyink to format codebase
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/428 from hsuyuming:fix_litellm_error_issue_427 dbec4949798e6399a0410d1b8ba7cc6a7cad7bdd
PiperOrigin-RevId: 754124679
--
16994cb2d5d646341f5285ca71d72697d81d18fe by Nilanjan De <nilanjan.de@gmail.com>:
chore: fix typos
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/272 from n1lanjan:fix-typos a1ab655b08ec08c5dd2da71aab9a2386e3610e84
PiperOrigin-RevId: 749690489
