encounter/adk-python
0
0
Fork 0
mirror of https://github.com/encounter/adk-python.git synced 2026-07-09 18:19:28 -07:00
Code Issues Packages Projects Releases Wiki Activity
740 Commits 12 Branches 39 Tags
0a65b528e3bdf570189d803a51e09a8c24b376c6
Commit Graph
23 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ankur SharmaandCopybara-Service 51be7a899c feat: Add implementation of BaseEvalService that runs evals locally 
This change:
- Introduces the LocalEvalService Class.
- Implements only the "perform_inference" method. Evaluate method will be implemented in the next CL.
- Adds required test coverage.

PiperOrigin-RevId: 781781954
 2025-07-10 19:25:24 -07:00
Ankur SharmaandCopybara-Service 0bd05df471 feat: Add Safety evaluator metric 
We add a new metric for evaluating safety of Agent's response to ADK Eval. We delegate the actual implementation to Vertex Gen AI Eval SDK, so using this metric will require GCP project.

As a part of this change, we created (refactored) a simple Facade for vertex gen ai eval sdk.

PiperOrigin-RevId: 778580406
 2025-07-02 11:30:31 -07:00
Ankur SharmaandCopybara-Service 62c4a85917 chore: Update ResponseEvaluator to use newer version of Eval SDK 
Also,
- removed functionality that was marked deprecated from the ResponseEvaluator class.
- Added unit test cases

PiperOrigin-RevId: 778568884
 2025-07-02 11:00:27 -07:00
Ankur SharmaandCopybara-Service b0d88bf172 feat: BaseEvalService declaration and surrounding data models 
Also, adds a metric registry.

PiperOrigin-RevId: 778186012
 2025-07-01 14:13:48 -07:00
Ankur SharmaandCopybara-Service 04de3e197d fix: Adding detailed information on each metric evaluation 
Additionally, few other small changes.
*   Updated a test fixture to support the latest eval data schema. Somehow I missed doing that previously.
*   Updated the `evaluation_generator.py` to use `run_async`, instead of `run`.
*   Also, raise an informed error when dependencies required eval are not installed.
*   Also, changed the behavior of AgentEvaluator.evaluate method to run all the evals, instead of failing at the first eval metric failure.

PiperOrigin-RevId: 775919127
 2025-06-25 18:32:02 -07:00
Ankur SharmaandCopybara-Service 984c1d6b8b test: Added unit test coverage for local_eval_set_results_manager.py 
PiperOrigin-RevId: 767202237
 2025-06-04 10:39:09 -07:00
Ankur SharmaandCopybara-Service 174afb3975 Added unit test coverage for local_eval_sets_manager.py 
PiperOrigin-RevId: 766392263
 2025-06-03 16:54:58 -07:00
Ankur SharmaandCopybara-Service a7575e078a feat: Added implementation for get_eval_case, update_eval_case and delete_eval_case for the local eval sets manager. 
PiperOrigin-RevId: 766383391
 2025-06-03 16:54:37 -07:00
Ankur SharmaandCopybara-Service e06e6753ad chore: Add the name of the agent as a billing label. This will help consumers slice their billing reports by Agent. 
Fixes https://github.com/google/adk-python/issues/452

PiperOrigin-RevId: 765402643
 2025-05-30 17:54:28 -07:00
Ankur SharmaandCopybara-Service 447798d573 Accounting for "references" to be absent in eval files in older format. 
PiperOrigin-RevId: 764454937
 2025-05-28 16:04:18 -07:00
Ankur SharmaandCopybara-Service 76116c8008 Minor documentation update to a file. 
PiperOrigin-RevId: 759742154
 2025-05-16 13:49:46 -07:00
Ankur SharmaandCopybara-Service 1c23556225 Updated test cases to use the new EvalSet schema to store test data. Also, added a utility to help migrate existing tests files to the new schema. 
Also, migrated existing test files to the new schema and deleted test session files as they are no longer needed.

PiperOrigin-RevId: 759318735
 2025-05-15 15:10:06 -07:00
Ankur SharmaandCopybara-Service 4c6820e78c Update AgentEvaluator to new new EvalSchema 
PiperOrigin-RevId: 759293759
 2025-05-15 14:09:23 -07:00
Ankur SharmaandCopybara-Service 95f994d7ca chore:add in memory session service to in memory runner. 
PiperOrigin-RevId: 759206471
 2025-05-15 10:35:21 -07:00
Ankur SharmaandCopybara-Service 3686608f8d Fixes test_cli_eval_success_path test. 
PiperOrigin-RevId: 758981214
 2025-05-14 22:20:59 -07:00
Ankur SharmaandCopybara-Service ada24d7171 Update Response Evaluators to use the new eval schema. 
PiperOrigin-RevId: 758929683
 2025-05-14 19:26:29 -07:00
Ankur SharmaandCopybara-Service ee674ce0ef Update Eval Run and TrajectoryEvaluator to use the new schema. 
PiperOrigin-RevId: 758927160
 2025-05-14 19:16:33 -07:00
Ankur SharmaandCopybara-Service 2cb74dd20e Update Eval Set Manager to use new EvalSet and EvalCase schema. 
PiperOrigin-RevId: 758921365
 2025-05-14 18:51:38 -07:00
Ankur SharmaandCopybara-Service cf06cc507a Refactor Eval Set Management into its own class. 
PiperOrigin-RevId: 758378377
 2025-05-13 14:32:38 -07:00
Ankur SharmaandCopybara-Service 82b8e304ec Convert Session To Invocation. 
PiperOrigin-RevId: 758356942
 2025-05-13 13:39:52 -07:00
Ankur SharmaandCopybara-Service 1237d5334f Adding Pydantic data models for eval set and eval case. 
PiperOrigin-RevId: 757920694
 2025-05-12 14:48:21 -07:00
Ankur SharmaandCopybara-Service 4289e02854 Update the 'type' value on the items/properties nested structures for Anthropic models to adhere to JSON schema. 
PiperOrigin-RevId: 754154596
 2025-05-02 15:30:44 -07:00
Ankur SharmaandCopybara-Service f872577f68 Currently if a model calls a FunctionTool without all the mandatory parameters, the code will just break. This change basically adds the capability for the FunctionTool to identify if the model is missing required arguments, and in that case, instead of breaking the execution, it provides a error message to the model so it could fix the request and retry. 
PiperOrigin-RevId: 751023475
 2025-04-24 09:31:40 -07:00