#rb swarm
[at]devin.doucette [at]ben.marsh
[FYI] antoine.guilbaud
#tests Booted editor and PIE'd into AthenaTest with UE-S3DataCachePath set to a different drive. Verified data is there.
#ROBOMERGE-SOURCE: CL 12474206 via CL 12474221 via CL 12474232 via CL 12474240
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v672-12450963)
[CL 12474252 by andrew grant in Main branch]
E.g.
'-ddc-local-missrate=50' simulates a 50% miss rate of keys on the local backend,
'-ddc-shared-misstypes=AnimSeq' simulates misses for any AnimSeq key. Use + to combine keys. E.g. AnimSeq+Audio
Memory DDC, FileSystemDDC and S3DDC support these options as do the classes that wrap other backends (e.g. AsyncPut).
#tests lots of warm and cold DDC
#swarm https://p4-swarm.epicgames.net/reviews/12458696
#ROBOMERGE-SOURCE: CL 12466119 via CL 12466177
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v672-12450963)
[CL 12466179 by andrew grant in Main branch]
E.g.
'-ddc-local-missrate=50' simulates a 50% miss rate of keys on the local backend,
'-ddc-shared-misstypes=AnimSeq' simulates misses for any AnimSeq key. Use + to combine keys. E.g. AnimSeq+Audio
Memory DDC, FileSystemDDC and S3DDC support these options as do the classes that wrap other backends (e.g. AsyncPut).
#tests lots of warm and cold DDC
[at]ben.marsh [at]josh.engebretson [at]devin.doucette
#ROBOMERGE-SOURCE: CL 12458693 via CL 12462167 via CL 12464485 via CL 12464606
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v672-12450963)
[CL 12464717 by andrew grant in Main branch]
#jira
#ROBOMERGE-SOURCE: CL 12455905 via CL 12455924 via CL 12455926 via CL 12460993 via CL 12461148
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v672-12450963)
[CL 12461272 by ben marsh in Main branch]
Also suppress some warning messages when a download is cancelled.
[FYI] Andrew.Grant
#jira
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: ben.marsh
#ROBOMERGE-SOURCE: CL 12453891 via CL 12454085 via CL 12454109 via CL 12459232 via CL 12459415
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v672-12450963)
[CL 12459547 by ben marsh in Main branch]
If a filesystem node is not available not prompt the user and optionally retry incase they need to mount a drive or start VPN
Fiilesystem nodes now perform a speed test using a selection of 'DDC sized' files to determine a classification (local, fast, ok, slow).
Add a new 'ConsiderSlowAt' property to the 'Filesystem' DDC node type. If latency to the node is >= this value then the node will be marked as slow which disables touch'ing and reduces file stats
Interface Changes
- Add the concept of a speed class to nodes
- Add GetName to nodes for better debugging / logging
- WouldCache query that allows caches to opt of of consideration early and avoid async tasks being created.
- Create a new 'FileBackedDerivedDataBackend' class that's the for the memory/boot backend and future classes
- TryToPrefetch interface functions for future use
Behavior Changes
- Moved parameter parsing into FileSysteDerivedDataBackend as things were getting out of hand
- FileSystemDerivedDataBackend now performs a speed test using 'DDC sized' files in separate directories and applies a classification
- Slow locations turn off touching of data on read
- Slow locations always return true for CachedDataProbablyExists. It's faster just to try to read and fail
- If the shared DDC is not available the user is prompted incase they need to mount it.
[at]ben.marsh [at]josh.engebretson
#rb swarm
#tests lots of PIE runs with / without this option
#ROBOMERGE-SOURCE: CL 12387516 via CL 12387517 via CL 12396622
#ROBOMERGE-BOT: (v671-12333473)
[CL 12396757 by andrew grant in Release-Engine-Staging branch]
#rb none
#jira
#ROBOMERGE-SOURCE: CL 12151719 via CL 12155008 via CL 12155068
#ROBOMERGE-BOT: (v659-12123632)
[CL 12155157 by ben marsh in Main branch]
#rb none
#rnx
#jira
#ROBOMERGE-SOURCE: CL 12147942 via CL 12147943 via CL 12149500
#ROBOMERGE-BOT: (v659-12123632)
[CL 12149604 by ben marsh in Main branch]
