The PR does two main things:
1) Introduces a new rubric based tool use metric
2) Given that we now have two rubric based metric, we refactor and create a new RubricBasedEvaluator interface.
PiperOrigin-RevId: 811983514
mainly because http://github.com/robots.txt disallows `/*/raw/` path. using GCS HTTP URIs is more reliable with Gemini model.
PiperOrigin-RevId: 811409688
Changes include:
- Implementing missing attributes. e.g. 'gen_ai.agent.name'
- Specifying reasons for not filling out some conditionally required attributes. e.g. 'gen_ai.data_source.id'
- Specifying reasons for not including certain attributes which are specified in current semconv. e.g. inference attributes on agent spans
PiperOrigin-RevId: 811379706
this is to allow turning on debug log for debugging if some unexpected behavior observed during running cache analysis experiments.
PiperOrigin-RevId: 811189954
This is to avoid serialization issue for some fields that are not json serializable.
meanwhile restructure the debug logs in context cache manager for better debugging potential issues.
PiperOrigin-RevId: 811182492
AppDetails require two pieces of information:
1) Instructions
2) Tools
Both these pieces of information are gathered using the llm_request that was passed to the model. This approach, slightly invasive, ensures that we capture the "exact" instructions and tools that were given to the model.
PiperOrigin-RevId: 811180648
Details:
1. Data model for storing App Details (the agentic system)
As we move towards LLM as Judge metrics, we see that some of these metrics need information about the Agentic system that was used for inferencing. We add a data model to capture that.
2. Data model for Steps
We refine the concept of intermediate data. Previously it stored data in the form of a multiple lists, thereby losing out on the chronological information. This information is needed for some of the metrics. So we refine the concept of intermediate data as series of logical steps that an Agent Take.
PiperOrigin-RevId: 811122784
