Commit Graph

11 Commits

Author SHA1 Message Date
devin doucette
6ab080890c DDC: Renamed cache request and response types from Chunk to GetChunk
#rb Zousar.Shaker
#rnx
#preflight 61e5ec410705ece855b8da74

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18635634 in //UE5/Release-5.0/... via CL 18635654 via CL 18635655
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18637582 by devin doucette in ue5-main branch]
2022-01-18 04:47:59 -05:00
zousar shaker
a455cb734d Ensure Jupiter DDC backend implements GetChunks in a functionally correct way.
Fix erroneous logging in Jupiter DDC backend when a cache record was missed.
Fix breakage in Jupiter DDC backend automated test caused by the header re-org.
Add GetChunks tests to Jupiter DDC backend automated test and increase the size and  byte variety for values in the test content.

#rb devin.doucette
#preflight 61e19ae5076be0fc4e128941

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 18618080 in //UE5/Release-5.0/... via CL 18618090 via CL 18618106
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18618116 by zousar shaker in ue5-main branch]
2022-01-14 10:53:17 -05:00
devin doucette
394fd0642b DDC: Made the backend namespaces consistent in advance of introducing ILegacyCacheStore
#rb Zousar.Shaker
#rnx
#preflight 61e04880250b9537f77549d7

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18597466 in //UE5/Release-5.0/... via CL 18597469 via CL 18597485
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v899-18417669)

[CL 18597488 by devin doucette in ue5-main branch]
2022-01-13 11:09:29 -05:00
devin doucette
b46e70ea4e DDC: Rename CompleteParams to Response and use FSharedString with no alias
#rb Zousar.Shaker
#rnx
#preflight 61ddf9e0f10add4574e6cf59

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18577467 in //UE5/Release-5.0/... via CL 18577471
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18577481 by devin doucette in ue5-release-engine-test branch]
2022-01-11 17:09:20 -05:00
devin doucette
23ddaa8bab DDC: Moved cache backends out of headers
This hides implementation details and reduces the number of files that need to change when refactoring backends.

Backends moved in this change: AsyncPut, Http, Memory, S3, Zen.

#rb Zousar.Shaker
#rnx
#preflight 61dd0d758d72a407aab89074

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18573340 in //UE5/Release-5.0/... via CL 18573347
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18573359 by devin doucette in ue5-release-engine-test branch]
2022-01-11 11:57:38 -05:00
zousar shaker
831400efc2 Add support for structured cache operations through the Jupiter DDC backend.
#rb devin.doucette
#preflight 61dc76dc2d5fd64ff7d4d4e8

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 18562740 in //UE5/Release-5.0/... via CL 18562761
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18562791 by zousar shaker in ue5-release-engine-test branch]
2022-01-10 13:43:40 -05:00
francis hurteau
86b3fd8de8 Refix crash in http request batch when the batch contains both a head and get request for the same cache keys
Added a test that could reliably reproduce the problem to validate it is properly fixed

#rb Danny.Couture, Zousar.Shaker
#preflight 610010b247c93a0001335319

#ROBOMERGE-OWNER: francis.hurteau
#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 16968978 via CL 16972104
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 16972120 by francis hurteau in ue5-release-engine-test branch]
2021-07-27 15:25:39 -04:00
danny couture
950c53fc2b Fix thread starvation for request in http backend by using a FIFO for thread waiting for a connection to be freed
Replace the sleep by a wait on an event triggered by the thread freeing the connection
Improve the test to output stats seen below and use the proper amount of threads on low core systems

Automation RunTests System.DerivedDataCache.HttpDerivedDataBackend.ConcurrentCachedDataProbablyExistsBatch

Before
   RPS: 238, AvgLatency: 90.09 ms, MaxLatency: 7.77 s
   RPS: 231, AvgLatency: 87.66 ms, MaxLatency: 7.77 s
   RPS: 245, AvgLatency: 139.07 ms, MaxLatency: 10.08 s

After
   RPS: 634, AvgLatency: 102.97 ms, MaxLatency: 0.26 s
   RPS: 653, AvgLatency: 102.51 ms, MaxLatency: 0.26 s
   RPS: 630, AvgLatency: 102.29 ms, MaxLatency: 0.26 s

#rnx
#rb Zousar.Shaker, Francis.Hurteau
#preflight 60ffe8317f21c90001f6116a

#ROBOMERGE-SOURCE: CL 16968010 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 16968015 by danny couture in ue5-release-engine-test branch]
2021-07-27 09:26:42 -04:00
francis hurteau
6862e14c30 Undo changelist 16938823 until investigation of cook failures are done
[FYI] Zousar.Shaker
#rb trivial
#preflight skip
discussed with Zoursar.Shaker, undo to fix cook failure

#ROBOMERGE-SOURCE: CL 16941401 via CL 16941404
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 16941410 by francis hurteau in ue5-release-engine-test branch]
2021-07-23 16:16:35 -04:00
francis hurteau
d7677fba02 Fix crash in http request batch when the batch contains both a head and get request for the same cache keys
Added a test that could reliably reproduce the problem to validate it is properly fixed
#preflight 60faeafc1f926d000113425b

#ROBOMERGE-SOURCE: CL 16938823 via CL 16938832
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 16938835 by francis hurteau in ue5-release-engine-test branch]
2021-07-23 13:15:05 -04:00
zousar shaker
1a835cd2c6 Fix bug where queued batch request failure in HttpDerivedDataBackend would cause incorrect DDC output (erroneous cache misses).
Adding basic unit test for HttpDerivedDataBackend.

#rb devin.doucette

#ROBOMERGE-SOURCE: CL 16885295 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16885298 by zousar shaker in ue5-release-engine-test branch]
2021-07-19 13:09:47 -04:00