encounter/adk-python
0
0
Fork 0
mirror of https://github.com/encounter/adk-python.git synced 2026-07-09 18:19:28 -07:00
Code Issues Packages Projects Releases Wiki Activity
699 Commits 12 Branches 39 Tags
438a008cb2ebbabd4b9c3da1b85cf93e781c23d0
Commit Graph
42 Commits
Author SHA1 Message Date
Ankur SharmaandCopybara-Service 0bd05df471 feat: Add Safety evaluator metric 
We add a new metric for evaluating safety of Agent's response to ADK Eval. We delegate the actual implementation to Vertex Gen AI Eval SDK, so using this metric will require GCP project.

As a part of this change, we created (refactored) a simple Facade for vertex gen ai eval sdk.

PiperOrigin-RevId: 778580406
 2025-07-02 11:30:31 -07:00
Ankur SharmaandCopybara-Service 62c4a85917 chore: Update ResponseEvaluator to use newer version of Eval SDK 
Also,
- removed functionality that was marked deprecated from the ResponseEvaluator class.
- Added unit test cases

PiperOrigin-RevId: 778568884
 2025-07-02 11:00:27 -07:00
Ankur SharmaandCopybara-Service b0d88bf172 feat: BaseEvalService declaration and surrounding data models 
Also, adds a metric registry.

PiperOrigin-RevId: 778186012
 2025-07-01 14:13:48 -07:00
Ankur SharmaandCopybara-Service 04de3e197d fix: Adding detailed information on each metric evaluation 
Additionally, few other small changes.
*   Updated a test fixture to support the latest eval data schema. Somehow I missed doing that previously.
*   Updated the `evaluation_generator.py` to use `run_async`, instead of `run`.
*   Also, raise an informed error when dependencies required eval are not installed.
*   Also, changed the behavior of AgentEvaluator.evaluate method to run all the evals, instead of failing at the first eval metric failure.

PiperOrigin-RevId: 775919127
 2025-06-25 18:32:02 -07:00
Joseph PagadoraandCopybara-Service 9597a446fd feat: Add rouge_score library to ADK eval dependencies, and implement RougeEvaluator that is computes ROUGE-1 for "response_match_score" metric 
PiperOrigin-RevId: 774949712
 2025-06-23 15:31:03 -07:00
Joseph PagadoraandCopybara-Service fa025d7559 feat: Add a new option eval_storage_uri in adk web & adk eval to specify GCS bucket to store eval data 
PiperOrigin-RevId: 774947795
 2025-06-23 15:24:57 -07:00
Google Team MemberandCopybara-Service 0a5cf45a75 feat: Implement GcsEvalSetResultsManager to handle storage of eval sets on GCS, and refactor eval set results manager 
Eval results will be stored as json files under `gs://{bucket_name}/{app_name}/evals/eval_history/`

PiperOrigin-RevId: 770499242
 2025-06-11 23:41:54 -07:00
Google Team MemberandCopybara-Service 1551bd4f4d feat: Re-factor some eval sets manager logic, and implement GcsEvalSetsManager to handle storage of eval sets on GCS 
Eval sets will be stored as json files under `gs://{bucket_name}/{app_name}/evals/eval_sets/`

PiperOrigin-RevId: 770487129
 2025-06-11 23:02:53 -07:00
Jason DaiandCopybara-Service 4942f19f3f chore: Remove duplicated model_config attribute in EvalMetric class 
PiperOrigin-RevId: 768214087
 2025-06-06 14:13:02 -07:00
Google Team MemberandCopybara-Service 068df04bce fix: import deprecated from typing_extensions 
Fixes https://github.com/google/adk-python/issues/1125

and add it to pyproject.toml. This was added to typing_extensions in 4.5.0. I followed versioning guidance from https://typing-extensions.readthedocs.io/en/latest/index.html#versioning-and-backwards-compatibility.

PiperOrigin-RevId: 767317467
 2025-06-04 15:51:43 -07:00
Ankur SharmaandCopybara-Service a7575e078a feat: Added implementation for get_eval_case, update_eval_case and delete_eval_case for the local eval sets manager. 
PiperOrigin-RevId: 766383391
 2025-06-03 16:54:37 -07:00
Garrett BischofandCopybara-Service 62d7bf58bb docs: change eval_dataset to eval_dataset_file_path_or_dir 
Copybara import of the project:

--
6596c477c7138df79d77e67181fd5979fd1a72e3 by Garrett Bischof <bischofgarrett@gmail.com>:

docs: change eval_dataset to eval_dataset_file_path_or_dir
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/397 from gwbischof:eval_dataset e8a0d0911c21ac76dab83e8520aaebc38d24adf5
PiperOrigin-RevId: 765638775
 2025-05-31 13:06:57 -07:00
Ankur SharmaandCopybara-Service 447798d573 Accounting for "references" to be absent in eval files in older format. 
PiperOrigin-RevId: 764454937
 2025-05-28 16:04:18 -07:00
Wei Sun (Jack)andCopybara-Service 19ca61d241 fix: model_config is misplaced for EvalMetricResultPerInvocation. 
PiperOrigin-RevId: 763648920
 2025-05-26 23:50:05 -07:00
Xiang (Sean) ZhouandCopybara-Service 7c2df7e4f6 refactor: refactor evaluation to make cli module depends on evaluation module. Modules outside of cli module should not reference cli module 
PiperOrigin-RevId: 763577749
 2025-05-26 19:18:55 -07:00
Xiang (Sean) ZhouandCopybara-Service be0786ea88 refactor: rename agent_dir to agents_dir and rename app_id to app_name in fast_api.py to make it consistent among every endpoints 
PiperOrigin-RevId: 763483339
 2025-05-26 11:51:19 -07:00
Google Team MemberandCopybara-Service 79681e3513 Write eval results locally from adk eval cli. 
PiperOrigin-RevId: 762499588
 2025-05-23 11:16:00 -07:00
Wei Sun (Jack)andCopybara-Service 1773cdab96 chore: Batch organize imports with isort. 
Command: `isort /src`
PiperOrigin-RevId: 761644934
 2025-05-21 13:23:32 -07:00
Google Team MemberandCopybara-Service 9928cafe32 Refactor eval results reporting with Eval Set Results manager. 
PiperOrigin-RevId: 761601525
 2025-05-21 11:30:40 -07:00
Google Team MemberandCopybara-Service 7d809eb9d9 Minor typo and grammar correction. 
PiperOrigin-RevId: 761119061
 2025-05-20 09:42:19 -07:00
Wei Sun (Jack)andCopybara-Service 0d7d7918b6 chore: logger = logging.getLogger(__name__) --> logger = logging.getLogger('google_adk.' + __name__) 
PiperOrigin-RevId: 759894901
 2025-05-16 23:04:02 -07:00
Ankur SharmaandCopybara-Service 76116c8008 Minor documentation update to a file. 
PiperOrigin-RevId: 759742154
 2025-05-16 13:49:46 -07:00
Google Team MemberandCopybara-Service 4d5760917d chore: Add alias_generators to new Eval pydantic models 
PiperOrigin-RevId: 759346603
 2025-05-15 16:21:20 -07:00
Ankur SharmaandCopybara-Service 1c23556225 Updated test cases to use the new EvalSet schema to store test data. Also, added a utility to help migrate existing tests files to the new schema. 
Also, migrated existing test files to the new schema and deleted test session files as they are no longer needed.

PiperOrigin-RevId: 759318735
 2025-05-15 15:10:06 -07:00
Ankur SharmaandCopybara-Service 4c6820e78c Update AgentEvaluator to new new EvalSchema 
PiperOrigin-RevId: 759293759
 2025-05-15 14:09:23 -07:00