The PR does two main things:
1) Introduces a new rubric based tool use metric
2) Given that we now have two rubric based metric, we refactor and create a new RubricBasedEvaluator interface.
PiperOrigin-RevId: 811983514
AppDetails require two pieces of information:
1) Instructions
2) Tools
Both these pieces of information are gathered using the llm_request that was passed to the model. This approach, slightly invasive, ensures that we capture the "exact" instructions and tools that were given to the model.
PiperOrigin-RevId: 811180648
Details:
1. Data model for storing App Details (the agentic system)
As we move towards LLM as Judge metrics, we see that some of these metrics need information about the Agentic system that was used for inferencing. We add a data model to capture that.
2. Data model for Steps
We refine the concept of intermediate data. Previously it stored data in the form of a multiple lists, thereby losing out on the chronological information. This information is needed for some of the metrics. So we refine the concept of intermediate data as series of logical steps that an Agent Take.
PiperOrigin-RevId: 811122784
For advanced eval use cases, we do expect agent developers to have rubrics that are specific to an Eval Case and in some cases even specific to a single invocation/turn in the eval case conversation.
A separate PR will be created to consume this data model changes in ADK Eval.
PiperOrigin-RevId: 805588808
Details:
- We plan on introducing Rubric based metrics in subsequent changes. This change introduces the data model needed that allows agent developer to provide rubrics.
- We also introduce a data model for the config that the eval system has been using for quite some time. It was loosely and informally described as a dictionary of metric names and expected thresholds. In this change, we actually formalize it using a pydantic data model, and extend it allow developers to specify rubrics as a part of their eval config.
What is a rubric based metric?
A rubric based metric is the assessment of a Agent's response (final or intermediate) along some rubric. This evaluation of agent's response significantly differs from the strategy where one has to provide a golden response.
PiperOrigin-RevId: 805488436
Merge https://github.com/google/adk-python/pull/2864
**Reason for this change:**
Multiple typos were found in comments, docstrings, and code throughout the codebase, which could lead to confusion and reduce code readability.
**Changes made:**
Fixed the following typos across 8 files:
1. contributing/samples/adk_answering_agent/utils.py:130: "extention" → "extension"
2. llms-full.txt:15171: "fuction" → "function"
3. src/google/adk/a2a/converters/part_converter.py:
- Line 96: "Conver" → "Convert", "reponse" → "response"
- Line 99: "suervice" → "service"
- Line 100: "accordinlgy" → "accordingly"
- Line 191: "Conver" → "Convert", "reponse" → "response"
- Line 195: "accordinlgy" → "accordingly"
4. src/google/adk/agents/base_agent.py:568: "custome" → "custom"
5. src/google/adk/evaluation/agent_evaluator.py:572: "Retruns" → "Returns"
6. src/google/adk/flows/llm_flows/basic.py:55: "outoput_schema" → "output_schema"
7. src/google/adk/flows/llm_flows/contents.py:136: "fuction_response" → "function_response"
8. src/google/adk/models/google_llm.py:138: "gemini_llm_connecton.py" → "gemini_llm_connection.py"
**Impact:**
This change will:
- Improve code documentation clarity and professionalism
- Make comments, docstrings, and code more readable and accurate
- Help prevent confusion for developers reading the code
- Ensure consistency in terminology throughout the codebase
This is a non-breaking change that only affects comments, documentation strings, and improves code clarity.
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/2864 from ammmr:chore-fix-typos 3cea9fcf6f21edb006b63e9258d2b82930dd961d
PiperOrigin-RevId: 805227784
Changed default values for `session_service`, `artifact_service`, and `run_config` from instances of mutable classes to `None`. Instances are now created within the function body if the argument is not provided, preventing unexpected shared state across function calls.
PiperOrigin-RevId: 804624564
Changed default values for `session_service`, `artifact_service`, and `run_config` from instances of mutable classes to `None`. Instances are now created within the function body if the argument is not provided, preventing unexpected shared state across function calls.
PiperOrigin-RevId: 804560641
This endpoint could be used by ADK Web to dynamically know:
- What are the available eval metrics in an App
- A description of those metrics
- A value range supported by those metrics
We also update the metric registry to make it mandatory to supply these details. The goal is to improve usability and interpretability of the eval metrics.
PiperOrigin-RevId: 787277695
Merge https://github.com/google/adk-python/pull/2148
This PR fixes#2071 exception string from `pip install google-adk[eval]` to `pip install "google-adk[eval]"` which makes it compatible for all the bash, zsh and other terminals
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/2148 from kavinkumar807:fix-module-not-found-exception-string-in-eval 914281006a0e162665c0933d0c0ee0c37eb397cf
PiperOrigin-RevId: 786752261
With this change we ensure that all three eval entry points, web, cli and pytest use the common LocalEvalService.
Updates to web and cli happened in a previous change.
PiperOrigin-RevId: 786445632
We update both adk web run eval endpoint and adk eval cli to use the LocalService. The old method is marked as deprecated and will be removed in later PRs.
PiperOrigin-RevId: 785612708
