Commit Graph

4 Commits

Author SHA1 Message Date
Devin Doucette
455b4ece9e DDC: Extended -DDC-MissTypes to match -DDC-Verify
In the examples below, <Node> is the name of a node in the cache store graph, or All to apply to every node.
-DDC-<Node>-MissTypes=StaticMesh+Texture will simulate a miss on every access to to static mesh and texture keys.
-DDC-<Node>-MissTypes=StaticMesh@12.5+Texture will simulate misses on 12.5% of accesses to static mesh keys and every access to texture keys.
-DDC-<Node>-MissRate=5 -DDC-<Node>-MissTypes=Texture will simulate misses on every access to texture keys and 5% of other keys.
-DDC-<Node>-MissRate=5 will simulate misses on 5% of keys.
-DDC-<Node>-MissSalt=PositiveInt32 will set the salt used to match keys to simulate misses on. A key always has the same simulated miss behavior with the same salt.

#preflight 6244d102b33098a72dc136df
#rb Zousar.Shaker

[CL 19572200 by Devin Doucette in ue5-main branch]
2022-03-31 10:06:47 -04:00
devin doucette
e3f0c1564c DDC: Fixed handling of metadata in the memory cache
#jira UE-146033
#preflight 62321041ac0173aa4131355a
#rb Matt.Peters, Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 19408655 in //UE5/Release-5.0/... via CL 19409276
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19410469 by devin doucette in ue5-main branch]
2022-03-16 15:31:59 -04:00
devin doucette
0e929ad2f3 DDC: Re-added the in-flight cache for async puts
- Legacy puts are now executing asynchronously.
- Memory cache now merges partial cache records.
- Memory cache is used to store data temporarily while it is being written by an async put.
- Expanded the deprecation of persisted boot/memory caches, which no longer offer the performance benefit that they used to.
- Fixed the pak file cache to skip data with a compressed size of over 2 GiB.

#jira UE-141307
#preflight 620d85f93609e19371510fb1
#lockdown Aurel.Cordonnier
#rb Zousar.Shaker
#rnx

#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-COMMAND: _robomerge UE5-Main
#ROBOMERGE-SOURCE: CL 19076205 in //UE5/Release-5.0/... via CL 19094550
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)

[CL 19095974 by devin doucette in ue5-main branch]
2022-02-23 13:40:18 -05:00
devin doucette
2800210897 DDC: Enabled compression of legacy cache data
- FileSystem, Http, Pak, S3 use the ValueWithLegacyFallback mode by default, which cause them to fall back to searching for uncompressed data if compressed data is not found.
- FileSystem has been fixed to store up to 1 MiB of compressed data inline with the value package rather than separately in content-addressable storage.
- Pak has been optimized to have GetChunks only load the required region of the requested value, rather than the whole value.
- Pak has been changed to stop storing data inline in the record package, since it will end up in the same file anyway when stored separately.
- Pak will upgrade the compressor and compression level when a compressed pak file is requested. Default cache compression uses Oodle Mermaid VeryFast and will upgrade to Oodle Kraken Optimal2.
- Zen does not have compression enabled by default, pending deployment of a new version that stores compressed values to Horde Storage.
- Added a missing request barrier when saving uncompressed data as compressed.

Example reduction in file system cache size when cooking for Windows:
- CitySample dropped from 66.5 GiB to 21.8 GiB.
- Lyra dropped from 2.54 GiB to 672 MiB.
- ShooterGame dropped from 1.21 GiB to 380 MiB.

Example reduction in compressed pak file cache size when cooking for Windows:
- CitySample dropped from 22.3 GiB to 18.5 GiB.
- Lyra dropped from 691 MiB to 543 MiB.
- ShooterGame dropped from 387 MiB to 313 MiB.

#jira UE-134381
#preflight 620a703f583261b0a658e043, 620a6fb2803d9066e6805310, 620a733117632e948459b6af
#lockdown Aurel.Cordonnier
#rb Zousar.Shaker

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18983671 in //UE5/Release-5.0/... via CL 18983890 via CL 18984096
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)

[CL 18984126 by devin doucette in ue5-main branch]
2022-02-14 14:43:39 -05:00