Ankur Sharma e88e667770 feat: Data model for Rubric based metric and eval config ...

Details:
- We plan on introducing Rubric based metrics in subsequent changes. This change introduces the data model needed that allows agent developer to provide rubrics.

- We also introduce a data model for the config that the eval system has been using for quite some time. It was loosely and informally described as a dictionary of metric names and expected thresholds. In this change, we actually formalize it using a pydantic data model, and extend it allow developers to specify rubrics as a part of their eval config.

What is a rubric based metric?
A rubric based metric is the assessment of a Agent's response (final or intermediate) along some rubric. This evaluation of agent's response significantly differs from the strategy where one has to provide a golden response.

PiperOrigin-RevId: 805488436

2025-09-10 13:20:07 -07:00

..

integration

docs: Clean up docs in sample

2025-08-22 11:36:58 -07:00

unittests

feat: Data model for Rubric based metric and eval config

2025-09-10 13:20:07 -07:00

__init__.py

Support async agent and model callbacks

2025-05-06 15:14:10 -07:00