Joseph PagadoraandCopybara-Service 6d2f33a59c feat: Update EvalConfig and EvalMetric data models to support custom metrics ...

Co-authored-by: Joseph Pagadora <jcpagadora@google.com>
PiperOrigin-RevId: 855517478

2026-01-12 21:37:21 -08:00

..

simulation

feat: Add custom instructions support to LlmBackedUserSimulator

2025-12-30 11:42:46 -08:00

__init__.py

…

mock_gcs_utils.py

…

test_app_details.py

…

test_eval_case.py

…

test_eval_config.py

feat: Update EvalConfig and EvalMetric data models to support custom metrics

2026-01-12 21:37:21 -08:00

test_evaluation_generator.py

chore: Move simulation related modules to a sub-package in evaluation

2025-12-01 13:16:12 -08:00

test_final_response_match_v1.py

chore: Introduce MetricInfoProvider interface, and refactor metric evaluators to use this interface to provide MetricInfo

2026-01-02 11:49:34 -08:00

test_final_response_match_v2.py

chore: Introduce MetricInfoProvider interface, and refactor metric evaluators to use this interface to provide MetricInfo

2026-01-02 11:49:34 -08:00

test_gcs_eval_set_results_manager.py

fix: Fix double JSON encoding when saving eval set results

2026-01-05 14:26:05 -08:00

test_gcs_eval_sets_manager.py

…

test_hallucinations_v1.py

…

test_in_memory_eval_sets_manager.py

…

test_llm_as_judge_utils.py

…

test_llm_as_judge.py

feat: Support per-eval case and per-invocation rubrics in rubric-based evaluators

2026-01-08 11:26:06 -08:00

test_local_eval_service.py

feat: Support per-eval case and per-invocation rubrics in rubric-based evaluators

2026-01-08 11:26:06 -08:00

test_local_eval_set_results_manager.py

fix: Fix double JSON encoding when saving eval set results

2026-01-05 14:26:05 -08:00

test_local_eval_sets_manager.py

…

test_metric_evaluator_registry.py

chore: Introduce MetricInfoProvider interface, and refactor metric evaluators to use this interface to provide MetricInfo

2026-01-02 11:49:34 -08:00

test_request_intercepter_plugin.py

…

test_response_evaluator.py

chore: Introduce MetricInfoProvider interface, and refactor metric evaluators to use this interface to provide MetricInfo

2026-01-02 11:49:34 -08:00

test_retry_options_utils.py

chore: Add default retry options as fall back to llm_request that are made during evals

2025-11-14 11:02:54 -08:00

test_rubric_based_evaluator.py

feat: Filter invocation and eval case rubrics by type when running rubric based evaluators

2026-01-08 12:23:33 -08:00

test_rubric_based_final_response_quality_v1.py

…

test_rubric_based_tool_use_quality_v1.py

chore: Introduce MetricInfoProvider interface, and refactor metric evaluators to use this interface to provide MetricInfo

2026-01-02 11:49:34 -08:00

test_safety_evaluator.py

chore: Introduce MetricInfoProvider interface, and refactor metric evaluators to use this interface to provide MetricInfo

2026-01-02 11:49:34 -08:00

test_trajectory_evaluator.py

fix: Allow string values for ToolTrajectoryCriterion.match_type

2026-01-05 12:43:53 -08:00

test_vertex_ai_eval_facade.py

…