devin doucette
402d04cde9
DDC: Added request name to FCacheStoreRequestStats
...
#jira UE-133395
#rb Zousar.Shaker
#rnx
[CL 26210120 by devin doucette in ue5-main branch]
2023-06-23 10:51:52 -04:00
devin doucette
3a730d179e
DDC: Added ICacheStoreStats to the Jupiter cache
...
#jira UE-133395
#rb Zousar.Shaker
#rnx
[CL 26186125 by devin doucette in ue5-main branch]
2023-06-22 12:55:58 -04:00
devin doucette
c4e4dfad44
DDC: Prepared the Jupiter cache for ICacheStoreStats
...
#jira UE-133395
#rb Zousar.Shaker
#rnx
[CL 26154477 by devin doucette in ue5-main branch]
2023-06-21 13:32:34 -04:00
kirill zorin
b6ee3a6c64
Fix UE_LOG callsites that have format string-related UB
...
#rb trivial
#preflight 647107780a6634dbb236a0a6
[CL 25651638 by kirill zorin in ue5-main branch]
2023-05-26 15:52:39 -04:00
joakim lindqvist
54660ae511
Added ability to override the http version used for the http cache using "UE-CloudDataCacheHttpVersion" env variable or "-CloudDataCacheHttpVersion" command line argument
...
#preflight 644b93ac09453df548007245
#rb zousar.shaker
[CL 25333909 by joakim lindqvist in ue5-main branch]
2023-05-04 09:31:50 -04:00
devin doucette
6448fc343a
DDC: Added EnvOAuthSecretOverride parameter to HttpCacheStore
...
#rb none
[FYI] Zousar.Shaker
[CL 25323458 by devin doucette in ue5-main branch]
2023-05-03 15:37:06 -04:00
devin doucette
5c3f44df32
DDC: Removed logging of the OAuth secret from HttpCacheStore
...
#rb none
#rnx
[CL 25323451 by devin doucette in ue5-main branch]
2023-05-03 15:36:58 -04:00
devin doucette
8ff0cd05ca
DDC: Added EHttpVersion to HttpClient
...
#rb Zousar.Shaker
#rnx
[CL 25184555 by devin doucette in ue5-main branch]
2023-04-25 13:54:42 -04:00
devin doucette
90200ff6d0
DDC: Fixes related to the use of OIDC functionality from DesktopPlatform
...
- Fixed a crash when HttpCacheStore is used without DesktopPlatform.
- Fixed GetOidcAccessToken and GetOidcTokenStatus failing for targets with no Intermediate directory.
- Added DesktopPlatform to DerivedDataTool to allow it to use HttpCacheStore with OIDC.
#rb Zousar.Shaker
[FYI] Dan.Engelbrecht
[CL 25072527 by devin doucette in ue5-main branch]
2023-04-17 15:23:51 -04:00
zousar shaker
3bd034edf7
Add commandline option to enable the "repair" mode for UE Cloud DDC client so that blobs that are missing get re-populated.
...
#rb devin.doucette
[CL 25027230 by zousar shaker in ue5-main branch]
2023-04-13 12:15:21 -04:00
zousar shaker
886b2fe9cd
Change HttpCacheStoreTest to pass access token rather than OAuth details to Zen intermediary instances. Also ensure that we don't overlap keys for Meta and NoMeta test data.
...
#rb joakim.lindqvist
[CL 24764335 by zousar shaker in ue5-main branch]
2023-03-23 13:05:19 -04:00
joakim lindqvist
40c47276bd
Error -> Warning when we are unable to resolve a token for Cloud DDC. This will not cause the build to fail, but it will typically be slower for it so error is harsh (as that in itself causes the build to fail)
...
[FYI] zousar.shaker
#preflight 63ef6b8b90198dffbae12467
[CL 24281972 by joakim lindqvist in ue5-main branch]
2023-02-17 10:25:12 -05:00
nicholas howe
a8ee4b6253
Fix RemoveTicker Hang in HttpCacheStore.
...
#jira UE-174489
#rb devin.doucette
#preflight 63c85026c246f14856661f40
[CL 23766649 by nicholas howe in ue5-main branch]
2023-01-18 17:59:46 -05:00
mark lintott
8bad2c0365
Removed the callback system from core Analytics system. Fixed up analytics gathering in DerivedDataCache, Virtualization and HttpCacheStore
...
#rb wes.hunt
#preflight 63be95bd68068a8bd6a4438e
[CL 23651011 by mark lintott in ue5-main branch]
2023-01-11 14:28:01 -05:00
Matt Peters
6f0b8a2906
HttpCacheStore: Fix backwards if statement for whether the compactbinary is valid in GetBodyAsString
...
#preflight 63b70052221aab21e01200c0
[CL 23589159 by Matt Peters in ue5-main branch]
2023-01-05 12:01:37 -05:00
matt peters
71802fb86c
Analytics: Backed out 22909100 and 22951911
...
#rnx
[CL 23583550 by matt peters in ue5-main branch]
2023-01-04 18:51:41 -05:00
Zousar Shaker
54d9057532
Fixes for UE Cloud DDC client:
...
- Print CbObject responses to the log/console as JSON, not raw binary
- Reduce the thresholds for connect timeout and slow transfer rate time window
- Instead of one request pool of size 128*2 requests, split GET and PUT operations into their own limited request pools and have the PUT request pool (size 20) be set smaller than the GET one (size 128)
- Stop using unbounded overflow for chained sequences of HTTP requests to avoid slow/failed requests due to many upload operations happening concurrently during high upload quantity scenarios (eg: shader version bump, editor domain version bump)
Testing:
- Cook of CitySample with shader version bumped to ensure there are no failed requests due to slow transfer
- Launch of CitySample editor with no local cache to observe throughput when all DDC data has to be fetched from upstream
#rb devin.doucette
#preflight 638c2d86cb3b754183f2e574
[CL 23385459 by Zousar Shaker in ue5-main branch]
2022-12-04 02:26:39 -05:00
Matt Peters
dd0f970188
HttpCacheStore: When a record is missing data and we return EStatus::Error and an empty record, fix the FCacheRecord passed into OnComplete to not have metadata or any values. The FCacheStoreHierarchy client of FHttpCacheStore assumes that metadata or values are only present on a successful record, and if it is present, incorrectly Puts the record as a valid record into the other cachestores in the hierarchy. This then breaks clients of DDC, because the record is invalid since it is missing some values.
...
This change fixes the warning messages when using EditorDomain: LogEditorDomain: Warning: Package %s received an invalid CacheRecord...
#rb Zousar.Shaker, Devin.Doucette
#rnx
#preflight 63753d95953c19d4351d8f11
[CL 23160500 by Matt Peters in ue5-main branch]
2022-11-16 15:12:51 -05:00
Devin Doucette
dcf192e97a
DDC: Remove any trailing slash from the Jupiter URI
...
#preflight 636d11ed0f51d9ed4afbf185
#rb Joakim.Lindqvist
#rnx
[CL 23079894 by Devin Doucette in ue5-main branch]
2022-11-10 10:57:37 -05:00
Zousar Shaker
ac3fc5db61
Print more detailed timing metrics for HTTP operations in the HttpCacheStore.
...
#rb devin.doucette
#preflight 636acc25dc30a4ce96bbe43c
[CL 23040720 by Zousar Shaker in ue5-main branch]
2022-11-08 16:52:32 -05:00
Devin Doucette
bfd3e68dc5
DDC: Fixed invalid put stats in the Jupiter client
...
#preflight 63691e65f56cab38c7c75f83
#rb Zousar.Shaker
#rnx
[CL 23010981 by Devin Doucette in ue5-main branch]
2022-11-07 10:33:06 -05:00
devin doucette
41c5373ebd
DDC: Fixed parameter names related to OIDC to be compliant with the coding standard and simplified analytics formatting
...
#preflight 636403b3de2c4dbb5d3a5b11
#rb Zousar.Shaker
#rnx
[CL 22968397 by devin doucette in ue5-main branch]
2022-11-03 18:00:06 -04:00
mark lintott
5ae432fc55
Addeed specific Unreal Cloud DDC analytics for core.cooking and core.loading events ( Domain, Effective Domain, Cache Hit/MIss for Cache Puts & Gets, FailedOIDC login attempts )
...
#rb joakim.lindqvist
#jira none
#preflight 6363c123d308b70e66f4cf33
[CL 22951911 by mark lintott in ue5-main branch]
2022-11-03 09:39:24 -04:00
devin doucette
271b9555dc
Added Unreal Cloud DDC to the base cache graphs
...
It is disabled by default outside of Epic, and can be enabled by configuring the Default key in the StorageServers section of the Engine config.
#rb Zousar.Shaker
[CL 22836839 by devin doucette in ue5-main branch]
2022-10-28 16:07:04 -04:00
Zousar Shaker
fee1c826d6
Update references to "Horde Storage" to "Unreal Cloud DDC" or appropriate variations.
...
Specifically:
HordeStorageServers-> StorageServers
HordeStorage -> UnrealCloudDDC
Horde Storage -> Unreal Cloud DDC
#rb matt.peters
#fyi devin.doucette
#fyi joakim.lindqvist
#jira UE-167262
#preflight 6349da9bad0f7e2f206125e3
[CL 22538004 by Zousar Shaker in ue5-main branch]
2022-10-14 18:39:39 -04:00
