diff --git a/contributing/samples/static_non_text_content/README.md b/contributing/samples/static_non_text_content/README.md index ae080f52..c84975a9 100644 --- a/contributing/samples/static_non_text_content/README.md +++ b/contributing/samples/static_non_text_content/README.md @@ -9,7 +9,7 @@ This sample demonstrates ADK's static instruction feature with non-text content - **Gemini Files API integration**: Demonstrates uploading documents and using file_data - **Mixed content types**: inline_data for images, file_data for documents - **API variant detection**: Different behavior for Gemini API vs Vertex AI -- **GCS file references**: Additional GCS file support when using Vertex AI +- **GCS file references**: Support for both GCS URI and HTTPS URL access methods in Vertex AI ## Static Instruction Content @@ -23,7 +23,7 @@ The agent includes: **Vertex AI:** 3. **Research paper**: Gemma research paper from Google Cloud Storage via GCS file reference -4. **Contributing guide**: Gemini Cookbook contributing guide from GitHub via HTTPS file reference +4. **AI research paper**: Same research paper accessed via HTTPS URL for comparison ## Content Used @@ -37,14 +37,14 @@ The agent includes: - Files are automatically cleaned up after 48 hours by the Gemini API **Vertex AI:** -- **Research Paper**: Gemma research paper (GCS file reference as `file_data`) - - Public GCS URI: `gs://cloud-samples-data/generative-ai/pdf/2403.05530.pdf` - - Demonstrates GCS file access in Vertex AI - - PDF format with technical AI research content -- **Contributing Guide**: Gemini Cookbook contributing guide (HTTPS file reference as `file_data`) - - Public GitHub URL: `https://raw.githubusercontent.com/google-gemini/cookbook/main/CONTRIBUTING.md` +- **Gemma Research Paper**: Research paper accessed via GCS URI (as `file_data`) + - GCS URI: `gs://cloud-samples-data/generative-ai/pdf/2403.05530.pdf` + - Demonstrates native GCS file access in Vertex AI + - PDF format with technical AI research content about Gemini 1.5 +- **AI Research Paper**: Same research paper accessed via HTTPS URL (as `file_data`) + - HTTPS URL: `https://storage.googleapis.com/cloud-samples-data/generative-ai/pdf/2403.05530.pdf` - Demonstrates HTTPS file access in Vertex AI - - Markdown format with development guidelines + - Agent can discover these are the same document and compare access methods ## Setup @@ -73,7 +73,9 @@ The agent will automatically load environment variables on startup. cd contributing/samples python -m static_non_text_content.main ``` -This runs 4 test prompts that specifically demonstrate the static content features. +This runs test prompts that demonstrate the static content features: +- **Gemini Developer API**: 4 prompts testing inline_data + Files API upload +- **Vertex AI**: 5 prompts testing inline_data + GCS/HTTPS file access comparison ### Interactive Mode ```bash @@ -101,13 +103,17 @@ The sample automatically runs test prompts when no `--prompt` is specified: **All API variants:** 1. "What reference materials do you have access to?" 2. "Can you describe the sample chart that was provided to you?" -3. "What does the contributing guide document say about best practices?" -4. "How do the inline image and file references in your instructions help you answer questions?" +3. "How do the inline image and file references in your instructions help you answer questions?" -**Vertex AI only (additional prompt):** +**Gemini Developer API only:** +4. "What does the contributing guide document say about best practices?" + +**Vertex AI only (additional prompts):** 5. "What is the Gemma research paper about and what are its key contributions?" +6. "Can you compare the research papers you have access to? Are they related or different?" -These prompts test `inline_data`, Files API `file_data` (Gemini API), and GCS/HTTPS `file_data` (Vertex AI). +**Gemini Developer API** tests: `inline_data` (image) + Files API `file_data` (uploaded document) +**Vertex AI** tests: `inline_data` (image) + GCS URI `file_data` + HTTPS URL `file_data` (same document via different access methods) ## How It Works diff --git a/contributing/samples/static_non_text_content/agent.py b/contributing/samples/static_non_text_content/agent.py index cda63846..58869155 100644 --- a/contributing/samples/static_non_text_content/agent.py +++ b/contributing/samples/static_non_text_content/agent.py @@ -75,7 +75,7 @@ def create_static_instruction_with_file_upload(): file_data_parts = [] if api_variant == GoogleLLMVariant.VERTEX_AI: - print("Using Vertex AI - adding GCS and GitHub file references") + print("Using Vertex AI - adding GCS URI and HTTPS URL references") # Add GCS file reference file_data_parts.append( @@ -90,20 +90,20 @@ def create_static_instruction_with_file_upload(): ) ) - # Add GitHub public file reference + # Add the same document via HTTPS URL to demonstrate both access methods file_data_parts.append( types.Part( file_data=types.FileData( - file_uri="https://raw.githubusercontent.com/google-gemini/cookbook/main/CONTRIBUTING.md", - mime_type="text/markdown", - display_name="Gemini Cookbook Contributing Guide", + file_uri="https://storage.googleapis.com/cloud-samples-data/generative-ai/pdf/2403.05530.pdf", + mime_type="application/pdf", + display_name="AI Research Paper (HTTPS)", ) ) ) additional_text = ( - " You also have access to the Gemma research paper from Google Cloud" - " Storage and the Gemini Cookbook contributing guide from GitHub." + " You also have access to a Gemma research paper from GCS" + " and an AI research paper from HTTPS URL." ) else: @@ -187,8 +187,8 @@ def create_static_instruction_with_file_upload(): instruction_text = """ When users ask questions, you should: 1. Use the reference chart above to provide context when discussing visual data or charts -2. Reference the Gemma research paper when discussing AI research, model architectures, or technical details -3. Reference the Gemini Cookbook contributing guide when explaining best practices and guidelines +2. Reference the Gemma research paper (from GCS) when discussing AI research, model architectures, or technical details +3. Reference the AI research paper (from HTTPS) when discussing research topics 4. Be helpful and informative in your responses 5. Explain how the provided reference materials relate to their questions""" else: diff --git a/contributing/samples/static_non_text_content/main.py b/contributing/samples/static_non_text_content/main.py index 5e786db0..1c9301c4 100644 --- a/contributing/samples/static_non_text_content/main.py +++ b/contributing/samples/static_non_text_content/main.py @@ -115,22 +115,33 @@ async def run_default_test_prompts(runner): app_name=APP_NAME, user_id=USER_ID ) - # Test prompts that specifically exercise the static content features + # Common test prompts for all API variants test_prompts = [ "What reference materials do you have access to?", "Can you describe the sample chart that was provided to you?", - "What does the contributing guide document say about best practices?", ( "How do the inline image and file references in your instructions " "help you answer questions?" ), ] - # Add Vertex AI specific prompt to test GCS file reference + # Add API-specific prompts if api_variant == GoogleLLMVariant.VERTEX_AI: + # Vertex AI has research papers instead of contributing guide + test_prompts.extend([ + ( + "What is the Gemma research paper about and what are its key " + "contributions?" + ), + ( + "Can you compare the research papers you have access to? Are they " + "related or different?" + ), + ]) + else: + # Gemini Developer API has contributing guide document test_prompts.append( - "What is the Gemma research paper about and what are its key " - "contributions?" + "What does the contributing guide document say about best practices?" ) for i, prompt in enumerate(test_prompts, 1):