In the examples below, <Node> is the name of a node in the cache store graph, or All to apply to every node.
-DDC-<Node>-MissTypes=StaticMesh+Texture will simulate a miss on every access to to static mesh and texture keys.
-DDC-<Node>-MissTypes=StaticMesh@12.5+Texture will simulate misses on 12.5% of accesses to static mesh keys and every access to texture keys.
-DDC-<Node>-MissRate=5 -DDC-<Node>-MissTypes=Texture will simulate misses on every access to texture keys and 5% of other keys.
-DDC-<Node>-MissRate=5 will simulate misses on 5% of keys.
-DDC-<Node>-MissSalt=PositiveInt32 will set the salt used to match keys to simulate misses on. A key always has the same simulated miss behavior with the same salt.
#preflight 6244d102b33098a72dc136df
#rb Zousar.Shaker
[CL 19572200 by Devin Doucette in ue5-main branch]
-DDC-Verify will verify determinism on every access to the cache.
-DDC-Verify=StaticMesh+Texture will verify determinism on every access static mesh and texture keys.
-DDC-Verify=StaticMesh@12.5+Texture will verify determinism on 12.5% of accesses to static mesh keys and every access to texture keys.
-DDC-VerifyRate=5 -DDC-Verify=Texture will verify detereminism on every access to texture keys and 5% of other keys.
-DDC-VerifyRate=5 -DDC-Verify will verify determinism on 5% of keys.
-DDC-VerifySalt=PositiveInt32 will set the salt used to match keys to verify. A key always has the same verification behavior with the same salt.
#preflight 6244ad99637925b5d3bc7fab
#rb Zousar.Shaker
[CL 19563192 by Devin Doucette in ue5-main branch]
- Added IRequestOwner::LaunchTask to launch a task as a request in the request owner.
- Renamed ExecuteInCacheThreadPool to LaunchTaskInCacheThreadPool.
- Made GCacheThreadPool private to one source file.
- Changed task scheduling throughout DDC to use these functions where appropriate.
#preflight 623b846d8900c14eecd4daa4
#rb Zousar.Shaker
[CL 19485686 by Devin Doucette in ue5-main branch]
Estimated to save 45s in a 1h50m cook of a larger internal project
#rb devin.doucette
#preflight 6238648a10251d53d55d3d12
[CL 19450563 by Johan Torp in ue5-main branch]
Edited the timing scopes to exclude invocation of the completion callback.
#preflight 62325890262232d86f812368
#rb Zousar.Shaker
#rnx
[CL 19419775 by Devin Doucette in ue5-main branch]
- Validate and shorten legacy keys lazily.
- Change to ValueOnly mode for PakFile and Zen.
#preflight 623235998a6037989395baf2
#rb Zousar.Shaker
#rnx
[CL 19410386 by Devin Doucette in ue5-main branch]
This would manifest as a cache entry being deemed to "exist", but later when you attempt to fetch it, it would not exist.
#rb devin.doucette
[FYI] joakim.lindqvist
#jira UE-145701
#lockdown simon.tourangeau
#preflight 6231f8866e25767a2189e99a
#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 19407779 in //UE5/Release-5.0/... via CL 19408966
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)
[CL 19409941 by zousar shaker in ue5-main branch]
This is being resolved by cherrypicking 19196840 by mark.lintott from UE5/Main (see original changelist details at the end of this cl description). On top of the cherrypick, there was a change made to re-use a single Http request when fetching stats. This avoids the overhead of doing name resolution repeatedly if the zen instance has a name instead of IP address.
Tested by:
- Running ShooterGame editor & observing UnrealInsights for ZenHttp_CurlPerform timers and ensuring they are not happening on the game thread when the editor is sitting idle (they are on background threads instead)
-Terminating ZenServer.exe while the editor is running and observing whether the framerate of the editor is negatively affected
#rb matt.peters
[FYI] mark.lintott
#jira UE-132849
#jira UE-143182
#lockdown aurel.cordonnier
#preflight 6220038831454c90cc13a123
Original changelist:
============
Blocking Zen HTTP stats request now runs async using futures
#rb Andriy.Tylychko
#[fyi] Zousar.Shaker
#jira UE-132849
#preflight 621e0c71e15c51d8c5bd5967
#ushell-cherrypick of 19196840 by mark.lintott
#ROBOMERGE-OWNER: zousar.shaker
#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 19234318 in //UE5/Release-5.0/... via CL 19236659
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)
[CL 19237357 by zousar shaker in ue5-main branch]
Also allow receipt of uncompressed buffers when using the inlining functionality to get values back in the first round trip for references.
#rb devin.doucette
#preflight 621fd1cf37049be51732f6a7
#jira UE-143616
#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 19227838 via CL 19228770 via CL 19228860 via CL 19228955 via CL 19229024
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)
[CL 19231554 by zousar shaker in ue5-main branch]
1) Increasing connection pool size
2) Allowing DNS and SSL/TLS data to be shared by connections
3) Switching to ValueOnly mode for Jupiter backends by default
4) Using "inline get" feature when querying Value data
Tested:
- .run editor -- -execCmds="Automation RunTests System.DerivedDataCache.HttpDerivedDataBackend.CacheStore;Quit" -NullRHI -stdout -unattended -ddc=DerivedDataBackendGraph_Jupiter
-Launch CitySample editor with empty local cache and open Small_City_LVL
-Launch CitySample editor with hot local cache and open Small_City_LVL
Can be improved further with a batched existence API for refs (instead of blobs). The current one does not work due to a Jupiter server bug, so this is an improvement that will have to come afterwards.
Note: There is a known issue involving Jupiter where the server will sometimes return uncompressed blobs. This manifests as cache misses due to invalid/rejected results returned from Jupiter. The client handles the miss and builds the data and stores it in its local cache but does not upload it to Jupiter (because Jupiter says it already has the blob). Subsequent runs will not suffer from the issue as the data will be present in the local cache. There are discussions in progress for how to address this issue on server or client side. No immediate mitigation is needed for it.
#rb devin.doucette
#jira UE-143616
#lockdown aurel.cordonnier
#preflight
#ROBOMERGE-OWNER: zousar.shaker
#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 19104205 via CL 19104462 via CL 19104579 via CL 19104585 via CL 19110006
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)
[CL 19113840 by zousar shaker in ue5-main branch]
- Legacy puts are now executing asynchronously.
- Memory cache now merges partial cache records.
- Memory cache is used to store data temporarily while it is being written by an async put.
- Expanded the deprecation of persisted boot/memory caches, which no longer offer the performance benefit that they used to.
- Fixed the pak file cache to skip data with a compressed size of over 2 GiB.
#jira UE-141307
#preflight 620d85f93609e19371510fb1
#lockdown Aurel.Cordonnier
#rb Zousar.Shaker
#rnx
#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-COMMAND: _robomerge UE5-Main
#ROBOMERGE-SOURCE: CL 19076205 in //UE5/Release-5.0/... via CL 19094550
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)
[CL 19095974 by devin doucette in ue5-main branch]
