The migration tool uses synchronous SQLAlchemy engines but users often provide async driver URLs (e.g., postgresql+asyncpg://) since that's what ADK requires at runtime.
This fix:
- Makes `to_sync_url()` public in `_schema_check_utils.py` for reuse
- Updates `migrate_from_sqlalchemy_pickle.py` to convert async URLs
- Updates `migrate_from_sqlalchemy_sqlite.py` to convert async URLs
- Adds comprehensive unit tests for `to_sync_url()` function
- Adds integration test for migration with async driver URLs
Fixes#4176
Co-authored-by: Liang Wu <wuliang@google.com>
PiperOrigin-RevId: 858359061
Merge https://github.com/google/adk-python/pull/3756
move event iteration inside api_client context in get_session
Move event iteration inside the api_client context manager in VertexAiSessionService.get_session() to prevent client closure during multi-page event fetching.
**Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.**
### Link to Issue or Description of Change
**1. Link to an existing issue (if applicable):**
- Closes: #3757
**2. Or, if no issue exists, describe the change:**
**Problem:**
When a session contains more than 100 events (requiring pagination), `VertexAiSessionService.get_session()` fails with:
```
RuntimeError: Cannot send a request, as the client has been closed.
```
The root cause is that the `events_iterator` is consumed **outside** the `async with self._get_api_client() as api_client:` context block. When the iterator needs to fetch page 2, 3, etc., the API client has already been closed because the `async with` block has exited.
```python
# Current buggy flow:
async with self._get_api_client() as api_client:
get_session_response, events_iterator = await asyncio.gather(...)
# ← Client closed here
async for event in events_iterator: # ← Fails on page 2+ (client closed)
session.events.append(...)
```
**Solution:**
Move the session creation, user validation, and event iteration **inside** the `async with` block so the API client remains open during the entire pagination process:
```python
async with self._get_api_client() as api_client:
get_session_response, events_iterator = await asyncio.gather(...)
# Validation and session creation...
async for event in events_iterator: # ← Now works for all pages
session.events.append(...)
# Client closed after all events are fetched
```
### Testing Plan
**Unit Tests:**
- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.
```bash
pytest tests/unittests/sessions/test_vertex_ai_session_service.py -v
```
**Added regression test:** `test_get_session_pagination_keeps_client_open`
- Creates a `MockAsyncClientWithPagination` that tracks whether it's inside the `async with` context
- Raises `RuntimeError` if iteration happens outside the context (matching real httpx behavior)
- Simulates 3 pages of events (100 + 100 + 50 = 250 events)
- Verifies all 250 events are successfully retrieved
**Manual End-to-End (E2E) Tests:**
1. Deploy an ADK agent to Vertex AI Agent Engine
2. Create a session and send 100+ messages to accumulate >100 events
3. Verify `get_session()` successfully retrieves all events without error
**Before fix:**
```
RuntimeError: Cannot send a request, as the client has been closed.
```
**After fix:**
- Session with 201 events (3 pages) loads successfully
- All events are retrieved and appended to the session
### Checklist
- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [x] I have manually tested my changes end-to-end.
- [x] Any dependent changes have been merged and published in downstream modules.
### Additional context
This bug affects any production deployment where users have extended conversations. Sessions accumulating >100 events (which triggers pagination) become completely unusable as the agent cannot load the session to process new messages.
The fix is minimal and maintains backward compatibility - it only changes the scope of the `async with` block without altering any logic or return values.
**Affected versions:** Tested on google-adk 1.19.0, but the bug exists in earlier versions as well.
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3756 from AlexisMarasigan:fix/vertex-ai-session-service-paginatio 01fbafa6524312f24f7c9feaffb07bff0ad49b77
PiperOrigin-RevId: 855451813
The fix will use quotes to escape "key", which is column name in the metadata table. Should work for different database types.
Merge https://github.com/google/adk-python/pull/4106
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/4106 from DineshThumma9:fix/mysql-reserved-keyword-issue e39d0d02f3695d6890bc3267417b5dad58f7e8ee
PiperOrigin-RevId: 854411915
The SqliteSessionService now accepts database paths in the form of SQLite URLs (e.g., "sqlite:///./sessions.db", "sqlite+aiosqlite:////absolute.db")
Close#4077
Co-authored-by: George Weale <gweale@google.com>
PiperOrigin-RevId: 853922433
Final part of https://github.com/google/adk-python/discussions/3605.
This change introduces:
- A new `adk migrate session` CLI command to run database schema upgrades.
- A migration script to upgrade from the old Pickle-based session schema (v0) to the new JSON-based schema (v1).
- A migration runner that orchestrates the upgrade process, handling sequential migrations and using temporary SQLite databases for intermediate steps if needed.
- Unit tests for the v0 to v1 migration.
Co-authored-by: Liang Wu <wuliang@google.com>
PiperOrigin-RevId: 852983323
The change updates the `StorageEvent.to_event` method to use `EventActions.model_validate` when rehydrating the `actions` field. This ensures that nested models within `EventActions`, such as `EventCompaction`, are correctly reconstructed from the stored data
Close#4047
Co-authored-by: George Weale <gweale@google.com>
PiperOrigin-RevId: 852408683
Part 2 of https://github.com/google/adk-python/discussions/3605.
The DatabaseSessionService now checks for the usage of a V1 schema based on the "adk_internal_metadata" table. Table creation and subsequent operations use either the V0 or V1 SQLAlchemy models accordingly. New databases will default to V1.
Migration script and CLI command will be provided in the next change.
Co-authored-by: Liang Wu <wuliang@google.com>
PiperOrigin-RevId: 845443406
Part 1 of https://github.com/google/adk-python/discussions/3605.
This change adds a new schema that uses JSON serialization to store Events data in the database. A new "adk_internal_metadata" table is also added to store information like schema version. Since we want to keep supporting existing DB, we fork from the original schema and call it "v0", while the new one is called "v1".
The change is no-op for existing users. In later change, the new schema will be used for new databases, and migration scripts will be provided for existing databases.
Co-authored-by: Liang Wu <wuliang@google.com>
PiperOrigin-RevId: 844986248
Merge https://github.com/google/adk-python/pull/3917
To migrate from existing DB :
ALTER TABLE events ALTER COLUMN error_message TYPE TEXT; -- PostgreSQL
ALTER TABLE events MODIFY error_message TEXT; -- MySQL
SQLite: Doesn't enforce VARCHAR length limits anyway. No impact.
### Link to Issue or Description of Change
**1. Link to an existing issue (if applicable):**
n/a
**2. Or, if no issue exists, describe the change:**
**Problem:**
When storing events with error messages longer than 1024 characters using `DatabaseSessionService`, PostgreSQL raises:
```
ERROR: value too long for type character varying(1024)
```
The `error_message` column in `StorageEvent` is defined as `String(1024)`, which maps to `VARCHAR(1024)`. Error messages can exceed 1024 characters.
**Solution:**
Change the column type from `String(1024)` to `Text` to allow unlimited length error messages.
### Testing Plan
**Unit Tests:**
- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.
$ pytest ./tests/unittests/sessions/ -v
======================= 75 passed, 3 warnings in 26.92s ========================
**Manual End-to-End (E2E) Tests:**
- Verified that events with long error messages (>1024 chars) can be stored in PostgreSQL
- Verified backward compatibility with existing databases
### Checklist
- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [x] I have manually tested my changes end-to-end.
- [x] Any dependent changes have been merged and published in downstream modules.
### Additional context
This is a minimal change (1 line) that only affects the `error_message` column type definition.
Co-authored-by: Xiang (Sean) Zhou <seanzhougoogle@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3917 from hiroakis:main 1474fd552cdbd7206de383e5507fd8a733aecda1
PiperOrigin-RevId: 844845692
Also provide a command line tool `adk migrate session` for DB migration
Addresses https://github.com/google/adk-python/discussions/3605
Addresses https://github.com/google/adk-python/issues/3681
To verify:
```
# Start one postgres DB
docker run --name my-postgres -d -e POSTGRES_DB=agent -e POSTGRES_USER=agent -e POSTGRES_PASSWORD=agent -e PGDATA=/var/lib/postgresql/data/pgdata -v pgvolume:/var/lib/postgresql/data -p 5532:5432 postgres
# Connect to an old version of ADK and produce some query data
adk web --session_service_uri=postgresql://agent:agent@localhost:5532/agent
# Check out to the latest branch and restart ADK web
# You should see error log ask you to migrate the DB
# Start a new DB
docker run --name migration-test-db \
-d \ --rm \ -e POSTGRES_DB=agent \ -e POSTGRES_USER=agent \ -e POSTGRES_PASSWORD=agent -e PGDATA=/var/lib/postgresql/data/pgdata -v migration_test_vol:/var/lib/postgresql/data -p 5533:5432 postgres
# DB Migration
adk migrate session \
--source_db_url="postgresql://agent:agent@localhost:5532/agent" \
--dest_db_url="postgresql://agent:agent@localhost:5533/agent"
# Run ADK web with the new DB
adk web --session_service_uri=postgresql+asyncpg://agent:agent@localhost:5533/agent
# You should see the data from old DB is migrated
```
Co-authored-by: Shangjie Chen <deanchen@google.com>
PiperOrigin-RevId: 837341139
The `database_session_service` now updates the `update_time` of a session to the event's timestamp when an event is appended
Close#2721
Co-authored-by: George Weale <gweale@google.com>
PiperOrigin-RevId: 834994070
This change introduces `input_transcription` and `output_transcription` fields to session events, enabling the storage and retrieval of transcription data in both the database and Vertex AI session services.
Closes#3172
Co-authored-by: Hangfei Lin <hangfei@google.com>
PiperOrigin-RevId: 834366848
Merge https://github.com/google/adk-python/pull/2889
# Implement Full async DatabaseSessionService
**Target Issue:** #1005
## Overview
This PR introduces an asynchronous implementation of the `DatabaseSessionService` with minimal breaking changes. The primary goal is to enable effective use of ADK in fully async environments and API endpoints while avoiding event loop blocking during database I/O operations.
## Changes
- Converted `DatabaseSessionService` to use async/await patterns throughout
## Testing Plan
The implementation has been tested following the project's contribution guidelines:
### Unit Tests
- All existing unit tests pass successfully
- Minor update to test requirements added to support `aiosqlite`
### Manual End-to-End Testing
- E2E tests performed using:
- **LLM Provider:** LiteLLM
- **Database:** PostgreSQL with `asyncpg` driver
```python
from google.adk.sessions.database_session_service import DatabaseSessionService
connection_string: str = (
"postgresql+asyncpg://PG_USER:PG_PSWD@PG_HOST:5432/PG_DB"
)
session_service: DatabaseSessionService = DatabaseSessionService(
db_url=connection_string
)
session = await session_service.create_session(
app_name="test_app", session_id="test_session", user_id="test_user"
)
assert session is not None
sessions = await session_service.list_sessions(app_name="test_app", user_id="test_user")
assert len(sessions.sessions) > 0
session = await session_service.get_session(
app_name="test_app", session_id="test_session", user_id="test_user"
)
assert session is not None
await session_service.delete_session(
app_name="test_app", session_id="test_session", user_id="test_user"
)
assert (
await session_service.get_session(
app_name="test_app", session_id="test_session", user_id="test_user"
)
is None
)
```
The implementation have been also tested using the following configurations for llm provider and Runner:
```python
def get_azure_openai_model(deployment_id: str | None = None) -> LiteLlm:
...
if not deployment_id:
deployment_id = os.getenv("AZURE_OPENAI_DEPLOYMENT_ID")
logger.info(f"Using Azure OpenAI deployment ID: {deployment_id}")
return LiteLlm(
model=f"azure/{os.getenv('AZURE_OPENAI_DEPLOYMENT_ID')}",
stream=True,
)
...
@staticmethod
def _get_runner(agent: Agent) -> Runner:
storage=DatabaseSessionService(db_url=get_pg_connection_string())
return Runner(
agent=agent,
app_name=APP_NAME,
session_service=storage,
)
...
async for event in self.runner.run_async(
user_id=user_id,
session_id=session_id,
new_message=content,
run_config=(
RunConfig(
streaming_mode=StreamingMode.SSE, response_modalities=["TEXT"]
)
if stream
else RunConfig()
),
):
last_event = event
if stream:
yield event
...
```
## Breaking Changes
- Database connection string format may need updates for async drivers
Co-authored-by: Shangjie Chen <deanchen@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/2889 from GitMarco27:feature/async_database_session_service e1b1b14934c1fb7975a6832cdd1549e94acab985
PiperOrigin-RevId: 830525148
The new Sqlite version has fixed schema and use a single column to store Event data, this should avoid DB migration for future add to the Event object.
- This change introduces `SqliteSessionService`, an asynchronous session service using `aiosqlite` that stores event data as JSON within SQLite.
- A migration script, `migrate_from_sqlalchemy_sqlite.py`, is included to transition data from the older SQLAlchemy-based SQLite schema to this new format.
- The CLI service registry is updated to use SqliteSessionService for sqlite:// URIs.
- Throw error when user trying to access a legacy DB and advice the user to do the migration.
Co-authored-by: Shangjie Chen <deanchen@google.com>
PiperOrigin-RevId: 829971174
This is about 35% decrease. This change refactors several ADK modules to import `vertexai` and its submodules only when they are first used, rather than at the top of the file. This improves module load times by avoiding unnecessary imports of large dependencies. Imports are also placed within `if TYPE_CHECKING:` blocks where appropriate.
Co-authored-by: Liang Wu <wuliang@google.com>
PiperOrigin-RevId: 829017293
