adk-python/tests/integration/fixture/bigquery_agent

Instructions

Run Evaluation

1. Set environment variables in your terminal:

export GOOGLE_GENAI_USE_VERTEXAI=FALSE
export GOOGLE_API_KEY=<your_api_key>
export GOOGLE_CLOUD_PROJECT=<your_bigquery_project>

1. Change to the current directory:

cd third_party/py/google/adk/tests/integration/fixture/bigquery_agent/

1. Customize the evaluation dataset to the environment GOOGLE_CLOUD_PROJECT by replacing the placeholder to the real project set in your environment:

sed -e "s:\${GOOGLE_CLOUD_PROJECT}:${GOOGLE_CLOUD_PROJECT}:g" simple.test.json -i

1. Run the following command as per https://google.github.io/adk-docs/evaluate/#3-adk-eval-run-evaluations-via-the-cli:

adk eval . simple.test.json --config_file_path=test_config.json

If it fails, re-run with --print_detailed_results flag to see more details on turn-by-turn evaluation.

Generate Evaluation dataset

1. Set environment variables in your terminal:

export GOOGLE_GENAI_USE_VERTEXAI=FALSE
export GOOGLE_API_KEY=<your_api_key>
export GOOGLE_CLOUD_PROJECT=<your_bigquery_project>

1. Set up google application default credentials on your machine.

gcloud auth application-default login

1. Change to the directory containing agent folder:

cd third_party/py/google/adk/tests/integration/fixture/

1. Run the following command to start the ADK web app:

adk web

1. Open the ADK web UI in your browser http://127.0.0.1:8000/dev-ui/?app=bigquery_agent.
2. Create an evaluation dataset by following these steps. This would generate file bigquery_agent/simple.evalset.json.
3. Note that this evaluation data would be tied to the agent interaction in the GOOGLE_CLOUD_PROJECT set in your environment. To normalize it by replacing the real project set in your environment to a placeholder, let's run the following command:

sed -e "s:${GOOGLE_CLOUD_PROJECT}:\${GOOGLE_CLOUD_PROJECT}:g"  bigquery_agent/simple.evalset.json > bigquery_agent/simple.test.json