Add queued work priority support to auto delete async tasks and schedule DDC Puts as Low priority
ThreadPool threads are now honoring the names they've been given on creation
Move the I/O only async DDC Get into the DDC ThreadPool with higher priority (Normal) than DCC Puts
#rb Devin.Doucette
[CL 15177897 by danny couture in ue5-main branch]
Added stats for Cloud based DCC ( ie Jupiter ) to DDC.Summary
Added BackEnd name to DDC.Summary
Added HasLocalDDC, HasSharedDDC, HasCloudDDC to DDC.Summary
[CL 15164347 by mark lintott in ue5-main branch]
Also made Jupiter a slow backend, as this generally is closer to the typical speed people have to it. We should make this automatic based on the latency we find from the health check call.
#rb devin.doucette
[CL 15154130 by Joakim Lindqvist in ue5-main branch]
ICache will eventually replace the existing cache access functions on FDerivedDataCacheInterface. The notable differences relative to the existing interface are:
- It is asynchronous with completion callbacks by default, and allows waiting when necessary.
- It allows batch requests containing multiple cache records by default.
- It uses a structured cache key made up of a bucket name and a hash.
- It supports prioritization and cancellation of requests.
- It supports structured cache records with metadata.
- It supports attachments to allow multi-part cache records.
- It uses a typed request object instead of an opaque integer handle.
- It provides cache policy flags to control how to query and store cache records, and which parts of a cache record to fetch.
This initial implementation of ICache is totally synchronous and uses the existing cache backends for storage.
#rb Zousar.Shaker
[CL 15152247 by Devin Doucette in ue5-main branch]
The async put backend now uses this new status to avoid calling CachedDataProbablyExists after every put, which has a high cost for certain backends.
#rb Zousar.Shaker
#fyi Joakim.Lindqvist, Stefan.Boberg
[CL 15000972 by Devin Doucette in ue5-main branch]
- The Io service currently returns response 204 when a HEAD request finds an entry, however the Europa service has been returning 200. The goal here is to update the DDC HEAD requests so that they will work with a response of 200 OR 204 so that we can unify the Jupiter apis.
- Added a utility method to FRequest that can be used to test if the response code is in the range that we consider a success.
- Updated FHttpDerivedDataBackend::CachedDataProbablyExists to use this.
- Updated the FRequest logging code to use this as well to avoid reporting successful operations as errors.
#rb Joakim.Lindqvist
#fyi Johan.Berg
#rnx
[CL 14940619 by paul chipchase in ue5-main branch]
For very long cooking sessions the Jupiter token could expire multiple times. That could cause the failed login counter to disable further login attempts. Fix this by resetting the counter on every successful login. Also reduced verbosity of 401 errors so that CI doesn't pick up expired tokens as errors.
#rb joakim.lindqvist
[CL 14887529 by Johan Berg in ue5-main branch]
Defer backfill up to date check to the async put as it may be expensive in the case of a shared ddc for example
This brings performance in line with working with the shared ddc disabled when the local cache is up to date.
Loading Apollo_Terrain_Edit: ~3m -> ~1m40s - DDC Access ~1m10s -> 4s when using the shared ddc
#rb Andrew.Grant, Devin.Doucette
[CL 14528803 by Francis Hurteau in ue5-main branch]
This will fix a good bunch of ForceRebuildPlatformData in UTexture2D::UpdateStreamingStatus because of the Boot DDC telling it has some mips cached effectively preventing other cache layers from receiving them.
When the Boot DDC is disabled after the editor boot, all the mips will go into oblivion, and nobody will have them anymore, causing stutters in the game-thread while streaming because a second full rebuild is now required.
Any DDC that plans on disappearing should never be alone in the cache hierarchy and should give other levels the opportunity to properly cache data before its disapperance.
#rb Francis.Hurteau, Devin.Doucette, Andrew.Grant
[CL 14268431 by danny couture in ue5-main branch]
Making this an error breaks progression of BuildCookRun scenarios under UAT.
I wasn't able to find a way to communicate arguments from UAT's command line to it's invocation of the cooker. The default behavior of performance liabilities shouldn't be a failed build. Ideally it would be optional, and there would be a feature to promote this warning to an error where it is specifically desired, perhaps when running in a data center.
[CL 14258264 by geoff evans in ue5-main branch]
