Commit Graph
2 Commits
Author SHA1 Message Date
Xiang (Sean) ZhouandCopybara-Service 9e0b1fb62b fix: Create context cache only when prefix matches with previous request
PiperOrigin-RevId: 817468275
2025-10-09 21:19:28 -07:00
Xiang (Sean) ZhouandCopybara-Service c66245a3b8 feat: support context caching
1. add a context cache config in app level which will apply to all agents in the app
2. pass on cache config through invocation context to llm_reqeust
3. store cache metadata in llm_response
4. lookup old cache metadata from latest event for reusing old cache
5. create new cache if old cache cannot be reused

PiperOrigin-RevId: 809158578
2025-09-19 13:17:02 -07:00