Any function that may create a request now has a IRequestOwner& parameter, and uses the Begin and End functions on the owner to manage the lifetime of any requests that it creates, as well as using End to invoke the completion callback for any request which has one.
The new FRequestBarrier may be used to block a group from being considered complete in a scope where more requests may be added to it.
#rb Matt.Peters
#rnx
#preflight 6109b5c403d303000144cce5
#preflight 610acf7103d30300016fda94
[CL 17060470 by Devin Doucette in ue5-main branch]
Added a test that could reliably reproduce the problem to validate it is properly fixed
#rb Danny.Couture, Zousar.Shaker
#preflight 610010b247c93a0001335319
#ROBOMERGE-OWNER: Francis.Hurteau
#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 16968978
#ROBOMERGE-BOT: (v838-16927207)
#ROBOMERGE-CONFLICT from-shelf
[CL 16972104 by Francis Hurteau in ue5-main branch]
Replace the sleep by a wait on an event triggered by the thread freeing the connection
Improve the test to output stats seen below and use the proper amount of threads on low core systems
Automation RunTests System.DerivedDataCache.HttpDerivedDataBackend.ConcurrentCachedDataProbablyExistsBatch
Before
RPS: 238, AvgLatency: 90.09 ms, MaxLatency: 7.77 s
RPS: 231, AvgLatency: 87.66 ms, MaxLatency: 7.77 s
RPS: 245, AvgLatency: 139.07 ms, MaxLatency: 10.08 s
After
RPS: 634, AvgLatency: 102.97 ms, MaxLatency: 0.26 s
RPS: 653, AvgLatency: 102.51 ms, MaxLatency: 0.26 s
RPS: 630, AvgLatency: 102.29 ms, MaxLatency: 0.26 s
#rnx
#rb Zousar.Shaker, Francis.Hurteau
#preflight 60ffe8317f21c90001f6116a
[CL 16968010 by danny couture in ue5-main branch]
Added a test that could reliably reproduce the problem to validate it is properly fixed
#preflight 60faeafc1f926d000113425b
#ROBOMERGE-SOURCE: CL 16938823
#ROBOMERGE-BOT: (v838-16927207)
[CL 16938832 by francis hurteau in ue5-main branch]
This allows the new cache to be implemented more efficiently using the old cache backends because functionality like the corruption wrapper and key length limiter can be bypassed and the individual backends can store cache records in whatever way is most efficient for them.
The hierarchical backend may request payloads when they are not required due to incomplete tracking of backend state, and GetPayload will never fill other backends due to the inefficiency of the existing backend framework.
The filesystem backend does not cache any state in memory, which makes requests for individual payloads less efficient than if it cached a mapping of cache payload key to raw hash after the first request for a cache key.
The HTTP, S3, and pak file backends are not implemented for the new interface.
The backends do not implement ICacheStore::CancelAll() because the existing backend framework provides WaitForQuiescence to wait for completion of async requests, and the implementation of ICacheStore by those backends works with that mechanism.
The non-leaf backends (hierarchical, async put, etc.) do not update stats from the ICacheStore functions.
#rb Zousar.Shaker
#rnx
#preflight 60899f35d324590001b47517
[CL 16148296 by Devin Doucette in ue5-main branch]
Still allows build machine runs to be thorough about completing PUTs.
Doesn't interrupt any in-flight PUTs, just prevents processing of any remaining queued PUTs.
#jira UE-112179
#rb devin.doucette
[CL 15920539 by Zousar Shaker in ue5-main branch]
