Commit Graph

4 Commits

Author SHA1 Message Date
Devin Doucette
d8c983bbe6 DDC: Optimized legacy cache key validation and shortening overhead
- Validate and shorten legacy keys lazily.
- Change to ValueOnly mode for PakFile and Zen.

#preflight 623235998a6037989395baf2
#rb Zousar.Shaker
#rnx

[CL 19410386 by Devin Doucette in ue5-main branch]
2022-03-16 15:28:29 -04:00
devin doucette
778a7d292b DDC: Added compression support to legacy cache values
The cache stores can individually control their "legacy mode" now, which offers a choice of ValueOnly, ValueWithLegacyFallback, LegacyOnly.
- Using ValueOnly will forward every legacy request through the Value API, which compresses values by default and supports values larger than 2 GiB.
- Using ValueWithLegacyFallback behaves like ValueOnly, but misses from GetValue are forwarded to GetCachedData, and stored with PutValue if found.
- Using LegacyOnly will forward every legacy request through the Legacy API.

FLegacyCacheValue uses shared state to ensure that each value is compressed or decompressed at most once.

#jira UE-134381
#preflight 62054b430c64e1822f41231b
#lockdown Mark.Lintott
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18940270 in //UE5/Release-5.0/... via CL 18941016 via CL 18941392
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)

[CL 18941401 by devin doucette in ue5-main branch]
2022-02-10 14:57:42 -05:00
devin doucette
b412a532ed DDC: Added MakeResponse(EStatus) to the cache request types
#rb Zousar.Shaker
#rnx
#preflight 61f1b88fc0033b39f89c8d84

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18744557 in //UE5/Release-5.0/... via CL 18744970 via CL 18745683
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18745958 by devin doucette in ue5-main branch]
2022-01-26 17:07:56 -05:00
devin doucette
3725815687 DDC: Added ILegacyCacheStore and folded the key length limit into the new FLegacyCacheKey
This API adds a little bit of overhead but, in exchange:
- Offers a path to migrate to the new Value API.
- Offers a path to provide compression while falling back to the previous format.
- Offers a path to backends batching requests without consuming a worker thread.
- Avoids copying the value multiple times during async puts.

#rb Zousar.Shaker
#rnx
#preflight 61e0a32fed50181feb57c5d6

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18607028 in //UE5/Release-5.0/... via CL 18607056 via CL 18607101
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18607126 by devin doucette in ue5-main branch]
2022-01-13 17:30:44 -05:00