Keyur JoshiandCopybara-Service 0918b647df fix: fix inconsistent method signatures for evaluate_invocations ...

The evaluate_invocations method override in Evaluator subclasses was not consistent, leading to errors during calls, especially when using kwargs. Made the overrides and calls consistent to resolve this issue.

Co-authored-by: Keyur Joshi <keyurj@google.com>
PiperOrigin-RevId: 850462752

2025-12-30 11:07:25 -08:00

..

simulation

feat: Introduce a post-hoc, per-turn evaluator for user simulations

2025-12-15 10:05:31 -08:00

__init__.py

No public description

2025-04-17 21:47:59 +00:00

mock_gcs_utils.py

feat: Implement GcsEvalSetResultsManager to handle storage of eval sets on GCS, and refactor eval set results manager

2025-06-11 23:41:54 -07:00

test_app_details.py

feat: Adds Rubric based final response evaluator

2025-09-24 20:30:51 -07:00

test_eval_case.py

feat: Adds Static User Simulator and User Simulator Provider

2025-10-21 11:15:11 -07:00

test_eval_config.py

fix: check if eval config file exists

2025-10-27 15:45:14 -07:00

test_evaluation_generator.py

chore: Move simulation related modules to a sub-package in evaluation

2025-12-01 13:16:12 -08:00

test_final_response_match_v1.py

feat: Added an Fast API new endpoint to serve eval metric info

2025-07-25 16:25:20 -07:00

test_final_response_match_v2.py

feat: Adds Rubric based final response evaluator

2025-09-24 20:30:51 -07:00

test_gcs_eval_set_results_manager.py

feat: Implement GcsEvalSetResultsManager to handle storage of eval sets on GCS, and refactor eval set results manager

2025-06-11 23:41:54 -07:00

test_gcs_eval_sets_manager.py

chore: Fix spelling

2025-11-03 13:33:53 -08:00

test_hallucinations_v1.py

chore: Fix spelling in tests

2025-11-06 11:21:59 -08:00

test_in_memory_eval_sets_manager.py

fix: Update create_eval_set API to return the created EvalSet and it route

2025-08-21 17:13:46 -07:00

test_llm_as_judge_utils.py

feat: Adds Rubric based final response evaluator

2025-09-24 20:30:51 -07:00

test_llm_as_judge.py

chore: Fix evaluation test cases to only use pytest features

2025-10-17 08:25:17 -07:00

test_local_eval_service.py

fix: fix inconsistent method signatures for evaluate_invocations

2025-12-30 11:07:25 -08:00

test_local_eval_set_results_manager.py

chore: Fix evaluation test cases to only use pytest features

2025-10-17 08:25:17 -07:00

test_local_eval_sets_manager.py

chore: Fix spelling

2025-11-03 13:33:53 -08:00

test_metric_evaluator_registry.py

feat: Added an Fast API new endpoint to serve eval metric info

2025-07-25 16:25:20 -07:00

test_request_intercepter_plugin.py

chore: Fix evaluation test cases to only use pytest features

2025-10-17 08:25:17 -07:00

test_response_evaluator.py

chore: Update vertexai & rouge scorer dependencies in eval to use a proxy

2025-10-22 22:44:53 -07:00

test_retry_options_utils.py

chore: Add default retry options as fall back to llm_request that are made during evals

2025-11-14 11:02:54 -08:00

test_rubric_based_evaluator.py

feat: Add Rubric based tool use metric

2025-09-26 15:47:42 -07:00

test_rubric_based_final_response_quality_v1.py

feat: Add Rubric based tool use metric

2025-09-26 15:47:42 -07:00

test_rubric_based_tool_use_quality_v1.py

feat: Add Rubric based tool use metric

2025-09-26 15:47:42 -07:00

test_safety_evaluator.py

chore: Update vertexai & rouge scorer dependencies in eval to use a proxy

2025-10-22 22:44:53 -07:00

test_trajectory_evaluator.py

feat: Added support for InOrder and AnyOrder match in ToolTrajectoryAvgScore Metric

2025-11-12 09:10:34 -08:00

test_vertex_ai_eval_facade.py

chore: Update vertexai & rouge scorer dependencies in eval to use a proxy

2025-10-22 22:44:53 -07:00