Also provide a command line tool `adk migrate session` for DB migration
Addresses https://github.com/google/adk-python/discussions/3605
Addresses https://github.com/google/adk-python/issues/3681
To verify:
```
# Start one postgres DB
docker run --name my-postgres -d -e POSTGRES_DB=agent -e POSTGRES_USER=agent -e POSTGRES_PASSWORD=agent -e PGDATA=/var/lib/postgresql/data/pgdata -v pgvolume:/var/lib/postgresql/data -p 5532:5432 postgres
# Connect to an old version of ADK and produce some query data
adk web --session_service_uri=postgresql://agent:agent@localhost:5532/agent
# Check out to the latest branch and restart ADK web
# You should see error log ask you to migrate the DB
# Start a new DB
docker run --name migration-test-db \
-d \ --rm \ -e POSTGRES_DB=agent \ -e POSTGRES_USER=agent \ -e POSTGRES_PASSWORD=agent -e PGDATA=/var/lib/postgresql/data/pgdata -v migration_test_vol:/var/lib/postgresql/data -p 5533:5432 postgres
# DB Migration
adk migrate session \
--source_db_url="postgresql://agent:agent@localhost:5532/agent" \
--dest_db_url="postgresql://agent:agent@localhost:5533/agent"
# Run ADK web with the new DB
adk web --session_service_uri=postgresql+asyncpg://agent:agent@localhost:5533/agent
# You should see the data from old DB is migrated
```
Co-authored-by: Shangjie Chen <deanchen@google.com>
PiperOrigin-RevId: 837341139
The `database_session_service` now updates the `update_time` of a session to the event's timestamp when an event is appended
Close#2721
Co-authored-by: George Weale <gweale@google.com>
PiperOrigin-RevId: 834994070
This change introduces `input_transcription` and `output_transcription` fields to session events, enabling the storage and retrieval of transcription data in both the database and Vertex AI session services.
Closes#3172
Co-authored-by: Hangfei Lin <hangfei@google.com>
PiperOrigin-RevId: 834366848
Merge https://github.com/google/adk-python/pull/2889
# Implement Full async DatabaseSessionService
**Target Issue:** #1005
## Overview
This PR introduces an asynchronous implementation of the `DatabaseSessionService` with minimal breaking changes. The primary goal is to enable effective use of ADK in fully async environments and API endpoints while avoiding event loop blocking during database I/O operations.
## Changes
- Converted `DatabaseSessionService` to use async/await patterns throughout
## Testing Plan
The implementation has been tested following the project's contribution guidelines:
### Unit Tests
- All existing unit tests pass successfully
- Minor update to test requirements added to support `aiosqlite`
### Manual End-to-End Testing
- E2E tests performed using:
- **LLM Provider:** LiteLLM
- **Database:** PostgreSQL with `asyncpg` driver
```python
from google.adk.sessions.database_session_service import DatabaseSessionService
connection_string: str = (
"postgresql+asyncpg://PG_USER:PG_PSWD@PG_HOST:5432/PG_DB"
)
session_service: DatabaseSessionService = DatabaseSessionService(
db_url=connection_string
)
session = await session_service.create_session(
app_name="test_app", session_id="test_session", user_id="test_user"
)
assert session is not None
sessions = await session_service.list_sessions(app_name="test_app", user_id="test_user")
assert len(sessions.sessions) > 0
session = await session_service.get_session(
app_name="test_app", session_id="test_session", user_id="test_user"
)
assert session is not None
await session_service.delete_session(
app_name="test_app", session_id="test_session", user_id="test_user"
)
assert (
await session_service.get_session(
app_name="test_app", session_id="test_session", user_id="test_user"
)
is None
)
```
The implementation have been also tested using the following configurations for llm provider and Runner:
```python
def get_azure_openai_model(deployment_id: str | None = None) -> LiteLlm:
...
if not deployment_id:
deployment_id = os.getenv("AZURE_OPENAI_DEPLOYMENT_ID")
logger.info(f"Using Azure OpenAI deployment ID: {deployment_id}")
return LiteLlm(
model=f"azure/{os.getenv('AZURE_OPENAI_DEPLOYMENT_ID')}",
stream=True,
)
...
@staticmethod
def _get_runner(agent: Agent) -> Runner:
storage=DatabaseSessionService(db_url=get_pg_connection_string())
return Runner(
agent=agent,
app_name=APP_NAME,
session_service=storage,
)
...
async for event in self.runner.run_async(
user_id=user_id,
session_id=session_id,
new_message=content,
run_config=(
RunConfig(
streaming_mode=StreamingMode.SSE, response_modalities=["TEXT"]
)
if stream
else RunConfig()
),
):
last_event = event
if stream:
yield event
...
```
## Breaking Changes
- Database connection string format may need updates for async drivers
Co-authored-by: Shangjie Chen <deanchen@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/2889 from GitMarco27:feature/async_database_session_service e1b1b14934c1fb7975a6832cdd1549e94acab985
PiperOrigin-RevId: 830525148
The new Sqlite version has fixed schema and use a single column to store Event data, this should avoid DB migration for future add to the Event object.
- This change introduces `SqliteSessionService`, an asynchronous session service using `aiosqlite` that stores event data as JSON within SQLite.
- A migration script, `migrate_from_sqlalchemy_sqlite.py`, is included to transition data from the older SQLAlchemy-based SQLite schema to this new format.
- The CLI service registry is updated to use SqliteSessionService for sqlite:// URIs.
- Throw error when user trying to access a legacy DB and advice the user to do the migration.
Co-authored-by: Shangjie Chen <deanchen@google.com>
PiperOrigin-RevId: 829971174
Merge https://github.com/google/adk-python/pull/3282
The `process_bind_param` and `process_result_value` methods in the `DynamicPickleType` class have been modified to handle MySQL dialect in addition to Spanner. This change ensures that pickled values are correctly processed for both database types.
**Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.**
### Link to Issue or Description of Change
**1. Link to an existing issue (if applicable):**
- Closes: #3283
**2. Or, if no issue exists, describe the change:**
_If applicable, please follow the issue templates to provide as much detail as
possible._
**Problem:**
When using `DatabaseSessionService` with MySQL backend in google-adk v1.17.0, the application crashes with the following error: app.resources.runner:event_generator:260 - Error in event_generator: (builtins.TypeError) 'tuple' object cannot be interpreted as an integer
<img width="1237" height="129" alt="image" src="https://github.com/user-attachments/assets/0a5fc223-600a-4a92-8443-4d37fb1267f6" />
Root cause: The `DynamicPickleType` class in `database_session_service.py` configures MySQL dialect to use `LONGBLOB` for storing pickled data (line 117-118), but the `process_bind_param` and `process_result_value` methods only handle pickle serialization/deserialization for Spanner dialect, not MySQL. This causes MySQL to attempt storing raw Python objects instead of pickled bytes, leading to serialization errors and potential data corruption.
**Solution:**
Added MySQL to the pickle serialization logic in both `process_bind_param` and `process_result_value` methods, treating it the same way as Spanner dialect. This ensures that:
- Data is properly pickled to bytes before being stored in MySQL's LONGBLOB column
- Data is properly unpickled when retrieved from the database
- No breaking changes to existing functionality for other dialects (SQLite, PostgreSQL)
### Testing Plan
_Please describe the tests that you ran to verify your changes. This is required
for all PRs that are not small documentation or typo fixes._
**Unit Tests:**
- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.
**Summary of `pytest` results:**
<img width="929" height="306" alt="image" src="https://github.com/user-attachments/assets/3d548b96-ac49-4101-8405-a289a722293c" />
**Manual End-to-End (E2E) Tests:**
_Please provide instructions on how to manually test your changes, including any
necessary setup or configuration. Please provide logs or screenshots to help
reviewers better understand the fix._
### Checklist
- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [x] I have manually tested my changes end-to-end.
- [x] Any dependent changes have been merged and published in downstream modules.
### Additional context
_Add any other context or screenshots about the feature request here._
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3282 from hung12ct:fix/mysql-pickle-serialization d9df37adb7dfbbfd8502a0fe65c4f8bca3d0d978
PiperOrigin-RevId: 825834360
We also change VertexAiSessionService and VertexAiMemoryBankService to both use keyword arguments for project, location, agent engine id, and express mode api key
PiperOrigin-RevId: 825719331
This CL refactors VertexAiSessionService to use the asynchronous aio client for all Vertex AI API calls. This ensures that the service methods are non-blocking and can be used effectively in an asyncio environment.
PiperOrigin-RevId: 824573356
