- Removed the "Cache" prefix from ECachePriority, FCacheRequest, ICacheRequest, FCachePayload, FCachePayloadId to allow reuse in the Build API.
- Added FCompressedBuffer as the starting point for the API for creating and accessing compressed buffers.
#rb Zousar.Shaker
#rnx
[CL 15511723 by Devin Doucette in ue5-main branch]
- FCacheRecord must now be created by FCacheRecordBuilder.
- FCacheRecord values are always an opaque buffer now, and ECacheRecordType has been removed.
- FCachePayload has been added as a common type for the cache record value and attachments.
- FCachePayloadId has been added as a 12-byte identifier for cache payloads.
- ICache::GetAttachments is now GetPayloads and can fetch both values and attachments.
- ECachePriority::Blocking has been added for requests that the caller will block on.
#rb Zousar.Shaker
#rnx
[CL 15471951 by Devin Doucette in ue5-main branch]
Add queued work priority support to auto delete async tasks and schedule DDC Puts as Low priority
ThreadPool threads are now honoring the names they've been given on creation
Move the I/O only async DDC Get into the DDC ThreadPool with higher priority (Normal) than DCC Puts
#rb Devin.Doucette
[CL 15177897 by danny couture in ue5-main branch]
Added stats for Cloud based DCC ( ie Jupiter ) to DDC.Summary
Added BackEnd name to DDC.Summary
Added HasLocalDDC, HasSharedDDC, HasCloudDDC to DDC.Summary
[CL 15164347 by mark lintott in ue5-main branch]
ICache will eventually replace the existing cache access functions on FDerivedDataCacheInterface. The notable differences relative to the existing interface are:
- It is asynchronous with completion callbacks by default, and allows waiting when necessary.
- It allows batch requests containing multiple cache records by default.
- It uses a structured cache key made up of a bucket name and a hash.
- It supports prioritization and cancellation of requests.
- It supports structured cache records with metadata.
- It supports attachments to allow multi-part cache records.
- It uses a typed request object instead of an opaque integer handle.
- It provides cache policy flags to control how to query and store cache records, and which parts of a cache record to fetch.
This initial implementation of ICache is totally synchronous and uses the existing cache backends for storage.
#rb Zousar.Shaker
[CL 15152247 by Devin Doucette in ue5-main branch]
Add some more verbose DDC logging
#rb none
#ROBOMERGE-OWNER: ben.ingram
#ROBOMERGE-AUTHOR: ben.ingram
#ROBOMERGE-SOURCE: CL 12905133 in //UE4/Release-4.25/... via CL 12905135 via CL 12905175
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v682-12900288)
[CL 12907642 by ben ingram in Main branch]
- VerifyDDC option is now correctly passed to the editor for editor-based tests and the cooker for cooked tests
- DDC= option added that allows kicked tests to use a different DDC backend (e.g. noshared). Defaults to 'default' which will use the default DDC set up.
[FYI] zabir.hoque
#jira nojira
#rb swarm
#lockdown cristina.riveron
#ROBOMERGE-SOURCE: CL 12726902 in //UE4/Release-4.25/... via CL 12726918 via CL 12781760
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v681-12776863)
[CL 12782565 by andrew grant in Main branch]
If a filesystem node is not available not prompt the user and optionally retry incase they need to mount a drive or start VPN
Fiilesystem nodes now perform a speed test using a selection of 'DDC sized' files to determine a classification (local, fast, ok, slow).
Add a new 'ConsiderSlowAt' property to the 'Filesystem' DDC node type. If latency to the node is >= this value then the node will be marked as slow which disables touch'ing and reduces file stats
Interface Changes
- Add the concept of a speed class to nodes
- Add GetName to nodes for better debugging / logging
- WouldCache query that allows caches to opt of of consideration early and avoid async tasks being created.
- Create a new 'FileBackedDerivedDataBackend' class that's the for the memory/boot backend and future classes
- TryToPrefetch interface functions for future use
Behavior Changes
- Moved parameter parsing into FileSysteDerivedDataBackend as things were getting out of hand
- FileSystemDerivedDataBackend now performs a speed test using 'DDC sized' files in separate directories and applies a classification
- Slow locations turn off touching of data on read
- Slow locations always return true for CachedDataProbablyExists. It's faster just to try to read and fail
- If the shared DDC is not available the user is prompted incase they need to mount it.
[at]ben.marsh [at]josh.engebretson
#rb swarm
#tests lots of PIE runs with / without this option
#ROBOMERGE-SOURCE: CL 12387516 via CL 12387517 via CL 12396622
#ROBOMERGE-BOT: (v671-12333473)
[CL 12396757 by andrew grant in Release-Engine-Staging branch]
