Estimated to save 45s in a 1h50m cook of a larger internal project
#rb devin.doucette
#preflight 6238648a10251d53d55d3d12
[CL 19450563 by Johan Torp in ue5-main branch]
Includes an optimization to avoid an extra copy of data loaded from the cache.
#jira UE-103039
#preflight 6205f379b84973a2bb76d942
#lockdown Aurel.Cordonnier
#rb Zousar.Shaker
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18954059 in //UE5/Release-5.0/... via CL 18954686 via CL 18955089
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)
[CL 18955195 by devin doucette in ue5-main branch]
SanitizeCacheKey and ValidateCacheKey should check the same condition
#rb devin.doucette, danny.couture
#ROBOMERGE-AUTHOR: charles.bloom
#ROBOMERGE-SOURCE: CL 18455963 in //UE5/Release-5.0/... via CL 18455975
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v898-18417669)
[CL 18455987 by charles bloom in ue5-release-engine-test branch]
- Implemented only for the structured cache initially.
- Records are stored as compact binary packages with payloads stored within the record if small enough.
- Records are stored in Records/{Bucket}/{01}/{23}/{456789abcdef0123456789abcdef01234567} based on the hash in the cache key.
- Content that exceeds the size allowed for storage within the record is stored in Content/{01}/{23}/{456789abcdef0123456789abcdef01234567} based on the hash of the content.
- Both records and content are written to disk with their hash as a suffix, and are always checked on load.
- Maximum size of compressed data stored in the package can be configured by MaxRecordSizeKB and MaxValueSizeKB.
- Maintenance of the cache, to delete unused files, has been rewritten and can now scan ~175k files/second on modern hardware, though is limited to 10k files/second in the config to avoid adding too much overhead.
#rb Zousar.Shaker
#preflight 618b75c14a0c609a29204032
#preflight 618bfb5ec717ba4e929b7ac0
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18133085 in //UE5/Release-5.0/... via CL 18133356
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v889-18060218)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 18133430 by devin doucette in ue5-release-engine-test branch]
The new API uses exported functions and cannot be included with the old API without compile errors in existing code that has an include-only dependency on DDC.
#rb Zousar.Shaker
#rnx
#preflight 610c01e3aeb05700011dc5ab
#ROBOMERGE-SOURCE: CL 17071263 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)
[CL 17071271 by devin doucette in ue5-release-engine-test branch]
Moved IDerivedDataCacheModule to its own header and renamed CreateOrGetDDC to CreateOrGetCache.
#rb Zousar.Shaker
#rnx
#preflight 6089d5a036afba00015c0348
[CL 16153121 by Devin Doucette in ue5-main branch]
ICache will eventually replace the existing cache access functions on FDerivedDataCacheInterface. The notable differences relative to the existing interface are:
- It is asynchronous with completion callbacks by default, and allows waiting when necessary.
- It allows batch requests containing multiple cache records by default.
- It uses a structured cache key made up of a bucket name and a hash.
- It supports prioritization and cancellation of requests.
- It supports structured cache records with metadata.
- It supports attachments to allow multi-part cache records.
- It uses a typed request object instead of an opaque integer handle.
- It provides cache policy flags to control how to query and store cache records, and which parts of a cache record to fetch.
This initial implementation of ICache is totally synchronous and uses the existing cache backends for storage.
#rb Zousar.Shaker
[CL 15152247 by Devin Doucette in ue5-main branch]
This is the preferred way to use string views. These interfaces were added in 4.25 so this is the time to fix them without breaking user code.
#jira none
#rb trivial
#rnx
#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 12236111 in //UE4/Release-4.25/... via CL 12236115
#ROBOMERGE-BOT: RELEASE (Release-4.25Plus -> Main) (v664-12234175)
[CL 12238549 by Devin Doucette in Main branch]
Updating the ArrayView with new comparison operators to have greater replacement for what TArray provides.
#rb Marc.Audy, Steve.Robb
#ROBOMERGE-OWNER: nick.darnell
#ROBOMERGE-AUTHOR: nick.darnell
#ROBOMERGE-SOURCE: CL 11843381 via CL 11843463 via CL 11843492
#ROBOMERGE-BOT: (v656-11643781)
[CL 11879995 by nick darnell in Main branch]
The existing GetSynchronous, GetAsynchronous, and Put functions are deprecated in favor of the DebugContext overloads. It is unlikely that a licensee has derived from FDerivedDataCacheInterface, but anyone who does will have a compile error that they override a non-virtual function and that they don't implement a pure virtual function, which will be straightforward to fix.
#rb Zousar.Shaker
#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 11210386 via CL 11210387
#ROBOMERGE-BOT: (v643-11205221)
[CL 11210609 by Devin Doucette in Main branch]
#rnx
#rb none
#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)
[CL 10870584 by ryan durand in Main branch]
