Commit Graph

10 Commits

Author SHA1 Message Date
devin doucette
3897ae4e92 DDC: Added ICacheStoreStats to the memory cache and cleaned up graph node lifetime
#jira UE-133395
#rb Zousar.Shaker
#rnx

[CL 25870464 by devin doucette in ue5-main branch]
2023-06-08 10:07:21 -04:00
devin doucette
4b200b0c7f DDC: Added -DDC-VerifyFix to overwrite existing records and values that differ from newly generated records and values
#rb Zousar.Shaker

[CL 24287758 by devin doucette in ue5-main branch]
2023-02-17 14:18:34 -05:00
Devin Doucette
1db832860f DDC: Fixed the default cache verify mode to verify every access
#preflight 63458481e76c1171e7f453a9
#rb Zousar.Shaker
#rnx

[CL 22459545 by Devin Doucette in ue5-main branch]
2022-10-11 12:10:29 -04:00
devin doucette
1d698f7814 DDC: Removed the remaining legacy cache implementations and made the default implementation non-virtual
#preflight 62a38afedf2d057cb2814ea4
#rb Steve.Robb
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 20601122 via CL 20601125 via CL 20601130
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v955-20579017)

[CL 20602835 by devin doucette in ue5-main branch]
2022-06-10 17:21:00 -04:00
charles bloom
7b58069f70 DDC Verify cache miss is just a message not a warning
#rb devin.doucette
#preflight none

[CL 19648508 by charles bloom in ue5-main branch]
2022-04-06 11:43:57 -04:00
charles bloom
9754c6cb2c Make DDC-Verify successes Verbose
#rb devin.doucette
#preflight none

[CL 19592154 by charles bloom in ue5-main branch]
2022-04-01 14:01:04 -04:00
Devin Doucette
455b4ece9e DDC: Extended -DDC-MissTypes to match -DDC-Verify
In the examples below, <Node> is the name of a node in the cache store graph, or All to apply to every node.
-DDC-<Node>-MissTypes=StaticMesh+Texture will simulate a miss on every access to to static mesh and texture keys.
-DDC-<Node>-MissTypes=StaticMesh@12.5+Texture will simulate misses on 12.5% of accesses to static mesh keys and every access to texture keys.
-DDC-<Node>-MissRate=5 -DDC-<Node>-MissTypes=Texture will simulate misses on every access to texture keys and 5% of other keys.
-DDC-<Node>-MissRate=5 will simulate misses on 5% of keys.
-DDC-<Node>-MissSalt=PositiveInt32 will set the salt used to match keys to simulate misses on. A key always has the same simulated miss behavior with the same salt.

#preflight 6244d102b33098a72dc136df
#rb Zousar.Shaker

[CL 19572200 by Devin Doucette in ue5-main branch]
2022-03-31 10:06:47 -04:00
Devin Doucette
cebf4161f9 DDC: Added -DDC-Verify arguments to control determinism verification
-DDC-Verify will verify determinism on every access to the cache.
-DDC-Verify=StaticMesh+Texture will verify determinism on every access static mesh and texture keys.
-DDC-Verify=StaticMesh@12.5+Texture will verify determinism on 12.5% of accesses to static mesh keys and every access to texture keys.
-DDC-VerifyRate=5 -DDC-Verify=Texture will verify detereminism on every access to texture keys and 5% of other keys.
-DDC-VerifyRate=5 -DDC-Verify will verify determinism on 5% of keys.
-DDC-VerifySalt=PositiveInt32 will set the salt used to match keys to verify. A key always has the same verification behavior with the same salt.

#preflight 6244ad99637925b5d3bc7fab
#rb Zousar.Shaker

[CL 19563192 by Devin Doucette in ue5-main branch]
2022-03-30 15:51:28 -04:00
devin doucette
778a7d292b DDC: Added compression support to legacy cache values
The cache stores can individually control their "legacy mode" now, which offers a choice of ValueOnly, ValueWithLegacyFallback, LegacyOnly.
- Using ValueOnly will forward every legacy request through the Value API, which compresses values by default and supports values larger than 2 GiB.
- Using ValueWithLegacyFallback behaves like ValueOnly, but misses from GetValue are forwarded to GetCachedData, and stored with PutValue if found.
- Using LegacyOnly will forward every legacy request through the Legacy API.

FLegacyCacheValue uses shared state to ensure that each value is compressed or decompressed at most once.

#jira UE-134381
#preflight 62054b430c64e1822f41231b
#lockdown Mark.Lintott
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18940270 in //UE5/Release-5.0/... via CL 18941016 via CL 18941392
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)

[CL 18941401 by devin doucette in ue5-main branch]
2022-02-10 14:57:42 -05:00
devin doucette
c7954aebce DDC: Switched to the async cache hierarchy and fully implemented the verify wrapper
The async cache hierarchy:
- Is required to add compression to the legacy cache by forwarding LegacyPut/LegacyGet to PutValue/GetValue.
- Is always present in the graph, unlike the previous cache hierarchy, which will allow significant simplification of leaf cache stores.
- Allows for the leaf cache store nodes to operate asynchronously without blocking a worker thread like the previous cache hierarchy.
- Shifts from controlling cache behavior by speed class to controlling cache behavior by local/remote classification, which is a critical distinction for a cache like Zen that can identify as both local and remote.
- Respects the local/remote and query/store flags in the cache policy.
- Does not fully implement the partial record cache policy at this time.
- Does not propagate records or values in GetChunks, which will be added in a future release.

The verify wrapper was previously missing an implementation for the new cache interface. This version is more efficient than the previous because requests through the new cache interface can compare data without loading it from storage, and load it only when a mismatch has been detected, to dump it to disk.

#jira UE-134381
#lockdown Mark.Lintott
#preflight 61fc32ac0a50c2606f266388
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18851861 in //UE5/Release-5.0/... via CL 18851943 via CL 18852169
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v910-18824042)

[CL 18852184 by devin doucette in ue5-main branch]
2022-02-03 16:49:32 -05:00