Commit Graph

98 Commits

Author SHA1 Message Date
Devin Doucette
c9aa2452df DDC: Allow both OAuthProvider and OAuthProviderIdentifier to be used in the same configuration for HttpCacheStore
- At least one of OAuthProvider, OAuthProviderIdentifier, or OAuthAccessTokenEnvOverride must be set.
- If only OAuthAccessTokenEnvOverride is set, then the environment variable must be set too.
- If only OAuthProvider is set, then OAuthSecret must be set too.
- If OAuthProvider is set, then OAuthClientId must be set.

#preflight 632d2eab10030508069b36c8
#rb Zousar.Shaker

[CL 22155720 by Devin Doucette in ue5-main branch]
2022-09-23 11:32:36 -04:00
devin doucette
1239a75353 DDC: Removed obsolete unstructured namespace parameter from the clients for Horde Storage and Zen
#rb Zousar.Shaker

[CL 22018759 by devin doucette in ue5-main branch]
2022-09-14 18:25:46 -04:00
Devin Doucette
9aee9a4bd8 DDC: Fixed unsafe type cast warnings
#preflight 6317fc04a20b67673b1de7c3
#rb Zousar.Shaker
#rnx

[CL 21850770 by Devin Doucette in ue5-main branch]
2022-09-07 10:37:16 -04:00
Joakim Lindqvist
99738a925d Allow users to interactivly login if they are missing a token. Will present a dialog for users if not unattended so they are aware they need to login.
#preflight https://horde.devtools.epicgames.com/job/62f26d7723003b62a3f3b58f

[CL 21334992 by Joakim Lindqvist in ue5-main branch]
2022-08-11 09:17:03 -04:00
Devin Doucette
1da9f58aa5 DDC: Optimized Horde Storage cache bucket transforms
#preflight 62dab595d54af4b9a23cea07
#rb Zousar.Shaker
#rnx

[CL 21221014 by Devin Doucette in ue5-main branch]
2022-07-22 10:54:37 -04:00
Devin Doucette
32b644e560 DDC: Use TSharedRef for shared Horde Storage completion functions
#preflight 62da25ec185da2495f7bbf5d
#rb Zousar.Shaker
#rnx

[CL 21220414 by Devin Doucette in ue5-main branch]
2022-07-22 10:21:26 -04:00
Devin Doucette
463ecf80f5 DDC: Added public key pinning support to the Horde Storage client
It would be better to use Certificate Transparency, but the UE SSL module currently expects pinned public keys, and verification using known key hashes is far better than no verification.

#preflight 62d5c582d54af4b9a23b92e7
#rb Zousar.Shaker
#rnx

[CL 21157474 by Devin Doucette in ue5-main branch]
2022-07-18 23:15:54 -04:00
Devin Doucette
c6bcab8076 DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC
- FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size.
- FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured.
- FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed.
- FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response.
- The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array.
- The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate.
- The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response.
- The response will be canceled automatically when destroyed through its THttpUniquePtr.
- The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue.
- The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed.

#preflight 62d580dd2e3e5993c3b678dd
#rb Zousar.Shaker

[CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
Devin Doucette
b7961682fd DDC: Fixed auth token refresh in HttpCacheStore to unregister from the ticker
Removed duplicate code, unnecessary scopes, and unused variables as well.

#preflight 62cda8de3d7ef9abccf7eb10
#rb Zousar.Shaker
#rnx

[CL 21061398 by Devin Doucette in ue5-main branch]
2022-07-12 15:36:38 -04:00
Joakim Lindqvist
bd3223cb4c Support for using OidcToken in editor / cooker to acquire a user token, currently opt in.
#preflight https://horde.devtools.epicgames.com/job/62c4108c8d5e678759f258f0

[CL 20964270 by Joakim Lindqvist in ue5-main branch]
2022-07-06 05:59:22 -04:00
Devin Doucette
646ddce8b0 DDC: Renamed HTTP enums to match modern terminology
#preflight 62bdb7c21c0b7587971c9f3c
#robomerge FNMain
#rb Zousar.Shaker
#rnx

[CL 20896677 by Devin Doucette in ue5-main branch]
2022-06-30 11:53:43 -04:00
devin doucette
2416a6e653 DDC: Fixed invalid or inconsistent whitespace in HttpCacheStore
#rb none
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 20882871 via CL 20885111 via CL 20885229
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v971-20777995)

[CL 20887593 by devin doucette in ue5-main branch]
2022-06-29 21:31:09 -04:00
devin doucette
1d698f7814 DDC: Removed the remaining legacy cache implementations and made the default implementation non-virtual
#preflight 62a38afedf2d057cb2814ea4
#rb Steve.Robb
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 20601122 via CL 20601125 via CL 20601130
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v955-20579017)

[CL 20602835 by devin doucette in ue5-main branch]
2022-06-10 17:21:00 -04:00
Devin Doucette
3490c0ee69 DDC: Restored the correct default of true for bResolveHostCanonicalName in HttpCacheStore
#preflight 629e348b5d78bd7cf1966b45
#rb trivial
#rnx
#fyi Zousar.Shaker

[CL 20523583 by Devin Doucette in ue5-main branch]
2022-06-06 13:33:27 -04:00
Devin Doucette
7d082d7531 DDC: Iteration on HttpClient
- Cleaned up some string allocations and conversions.
- Changed code formatting for greater consistency with DDC.

#preflight 62992f72f03d578360b2bfec
#rb Zousar.Shaker
#rnx

[CL 20485244 by Devin Doucette in ue5-main branch]
2022-06-03 09:12:05 -04:00
Ben Marsh
8da76cf563 Fix non-unity error.
#preflight none

[CL 20469945 by Ben Marsh in ue5-main branch]
2022-06-02 11:06:47 -04:00
Devin Doucette
5c2b7f7381 DDC: Temporary compile fix for platforms that should not include DerivedDataCache
#preflight skip
#rb none
#rnx

[CL 20445864 by Devin Doucette in ue5-main branch]
2022-05-31 23:25:37 -04:00
Devin Doucette
1bbb384300 DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen
#preflight 62967fd60c151996e13038ec
#rb Zousar.Shaker

[CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
Zousar Shaker
edf8ae1f0b 2nd submit:
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath.  This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store.  Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.

Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298
#preflight 629642a1be0ae0b33a46d012

[CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
mark lintott
da46ade47c Added Cache Speed Stats to Analytics
Added Zen Server Stats to Analytics
#rb zousar.shaker, devin.doucette
#jira none
#preflight 6295d84bd57da28cc8ee9eab

[CL 20434901 by mark lintott in ue5-main branch]
2022-05-31 05:11:05 -04:00
Zousar Shaker
77f51a02b9 [Backout] - CL20433163
#fyi Zousar.Shaker
Original CL Desc
-----------------------------------------------------------------
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath.  This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store.  Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.

Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298

[CL 20433363 by Zousar Shaker in ue5-main branch]
2022-05-30 19:32:59 -04:00
Zousar Shaker
2f6cbf5012 Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.
Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298

[CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
David Harvey
b5840fed7f removing several PLATFORM_HOLOLENS references, mostly around WindowsHWrapper etc.
#jira UE-149794
#rnx
#preflight 628f7a4df622d972b5db8318
#rb pending

[CL 20391626 by David Harvey in ue5-main branch]
2022-05-27 07:04:22 -04:00
Zousar Shaker
56023240e1 Cleanup and private detail pass on Http Client code within DDC in preparation for re-use from multiple cache stores.
#rb devin.doucette
#preflight 628fd292f622d972b5f27a6c

[CL 20386642 by Zousar Shaker in ue5-main branch]
2022-05-26 18:07:41 -04:00
Zousar Shaker
b0c4a2516e Fix incorrect handling of empty "needs" list when performing existence check in Horde Storage.
#rb devin.doucette
#fyi paul.chipchase
#preflight 6287ac2c2c34da686ae7c760

[CL 20295308 by Zousar Shaker in ue5-main branch]
2022-05-20 11:32:33 -04:00