Commit Graph

30 Commits

Author SHA1 Message Date
Zousar Shaker
edf8ae1f0b 2nd submit:
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath.  This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store.  Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.

Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298
#preflight 629642a1be0ae0b33a46d012

[CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
mark lintott
da46ade47c Added Cache Speed Stats to Analytics
Added Zen Server Stats to Analytics
#rb zousar.shaker, devin.doucette
#jira none
#preflight 6295d84bd57da28cc8ee9eab

[CL 20434901 by mark lintott in ue5-main branch]
2022-05-31 05:11:05 -04:00
Zousar Shaker
77f51a02b9 [Backout] - CL20433163
#fyi Zousar.Shaker
Original CL Desc
-----------------------------------------------------------------
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath.  This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store.  Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.

Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298

[CL 20433363 by Zousar Shaker in ue5-main branch]
2022-05-30 19:32:59 -04:00
Zousar Shaker
2f6cbf5012 Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.
Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298

[CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
David Harvey
b5840fed7f removing several PLATFORM_HOLOLENS references, mostly around WindowsHWrapper etc.
#jira UE-149794
#rnx
#preflight 628f7a4df622d972b5db8318
#rb pending

[CL 20391626 by David Harvey in ue5-main branch]
2022-05-27 07:04:22 -04:00
Zousar Shaker
56023240e1 Cleanup and private detail pass on Http Client code within DDC in preparation for re-use from multiple cache stores.
#rb devin.doucette
#preflight 628fd292f622d972b5f27a6c

[CL 20386642 by Zousar Shaker in ue5-main branch]
2022-05-26 18:07:41 -04:00
Zousar Shaker
b0c4a2516e Fix incorrect handling of empty "needs" list when performing existence check in Horde Storage.
#rb devin.doucette
#fyi paul.chipchase
#preflight 6287ac2c2c34da686ae7c760

[CL 20295308 by Zousar Shaker in ue5-main branch]
2022-05-20 11:32:33 -04:00
Zousar Shaker
f263b7ed54 2nd check-in:
Separate re-usable HTTP types from the HttpCacheStore to allow for re-use in the future by other cache stores.

#rb devin.doucette
#rnx
#preflight 62841cbdf536779152dfc561

[CL 20260417 by Zousar Shaker in ue5-main branch]
2022-05-18 09:58:59 -04:00
Zousar Shaker
b4cb8da065 [Backout] - CL20252872
#fyi Zousar.Shaker
Original CL Desc
-----------------------------------------------------------------
Separate re-usable HTTP types from the HttpCacheStore to allow for re-use in the future by other cache stores.

#rb devin.doucette
#rnx
#preflight 62841cbdf536779152dfc561

[CL 20254560 by Zousar Shaker in ue5-main branch]
2022-05-17 20:51:15 -04:00
Zousar Shaker
8805f32c05 Separate re-usable HTTP types from the HttpCacheStore to allow for re-use in the future by other cache stores.
#rb devin.doucette
#rnx
#preflight 62841cbdf536779152dfc561

[CL 20252872 by Zousar Shaker in ue5-main branch]
2022-05-17 18:36:19 -04:00
Zousar Shaker
d08f81b885 Ensure that http headers can be found in a case-insensitive way as expected by the callers of GetHeader.
#rb devin.doucette
#rnx
#preflight 627b53295b58374330b5484c
#preflight 627bf3ce0a5817c9d952fc57

[CL 20144831 by Zousar Shaker in ue5-main branch]
2022-05-11 13:52:26 -04:00
Devin Doucette
5021354ab8 DDC: Fixed HttpCacheStore to shut down its thread
FHttpSharedData is no longer constructed as a function-local static because that makes shutdown more difficult to handle correctly. The cache store now owns the shared data and destroys it when it shuts down.

Includes greater encapsulation of the shared data and migration from FRunnable/FRunnableThread to FThread.

#preflight 627aad2bc42338be65272366
#rb Zousar.Shaker
#rnx

[CL 20138109 by Devin Doucette in ue5-main branch]
2022-05-11 09:10:24 -04:00
Zousar Shaker
4ee0c804e0 [Backout] - CL20029407
#fyi Zousar.Shaker
Original CL Desc
-----------------------------------------------------------------
Limit concurrent connections when using the Curl multi API for Jupiter DDC operations.

#rb devin.doucette
#preflight 627040b4645c64f3a25a14ca
#preflight 62716d2c9d6c2f8f5b2558f2

[CL 20057204 by Zousar Shaker in ue5-main branch]
2022-05-05 09:58:02 -04:00
Zousar Shaker
5127ed3cb9 Limit concurrent connections when using the Curl multi API for Jupiter DDC operations.
#rb devin.doucette
#preflight 627040b4645c64f3a25a14ca
#preflight 62716d2c9d6c2f8f5b2558f2

[CL 20029407 by Zousar Shaker in ue5-main branch]
2022-05-03 14:23:07 -04:00
Zousar Shaker
05664ce5b2 Avoid querying Horde Storage during GetChunks calls for "Null hash" compressed blob value when you've experienced a miss for the record it belongs to.
#rb devin.doucette
#preflight 626ff4878063424f88a90ceb

[CL 20009546 by Zousar Shaker in ue5-main branch]
2022-05-02 11:52:58 -04:00
Devin Doucette
a99439300d DDC: Added and clarified wait timing scopes in the Horde Storage cache client
#preflight 626ae32e836be6e73ac9a190
#rb Zousar.Shaker
#rnx

[CL 19967049 by Devin Doucette in ue5-main branch]
2022-04-28 15:29:34 -04:00
Devin Doucette
92a888c1f6 Replaced uses of _ASV with ANSITEXTVIEW
#preflight 6266acf50634d0904cdfe91c
#rb Steve.Robb
#rnx

[CL 19903251 by Devin Doucette in ue5-main branch]
2022-04-25 11:31:36 -04:00
Zousar Shaker
28c6283a57 2nd submit with fix for lifetime issue:
Allow async operations for Jupiter DDC backend.

#rb devin.doucette
#jira UE-144412
#preflight 624c5fb68d1db441a90f23fe

[CL 19631067 by Zousar Shaker in ue5-main branch]
2022-04-05 16:43:41 -04:00
Zousar Shaker
f34273a349 [Backout] - CL19628588
#fyi Zousar.Shaker
Original CL Desc
-----------------------------------------------------------------
Allow async operations for Jupiter DDC backend.

#rb devin.doucette
#jira UE-144412
#preflight 624c5fb68d1db441a90f23fe

[CL 19629334 by Zousar Shaker in ue5-main branch]
2022-04-05 15:17:17 -04:00
Zousar Shaker
56e179a5f4 Allow async operations for Jupiter DDC backend.
#rb devin.doucette
#jira UE-144412
#preflight 624c5fb68d1db441a90f23fe

[CL 19628588 by Zousar Shaker in ue5-main branch]
2022-04-05 14:18:39 -04:00
Devin Doucette
455b4ece9e DDC: Extended -DDC-MissTypes to match -DDC-Verify
In the examples below, <Node> is the name of a node in the cache store graph, or All to apply to every node.
-DDC-<Node>-MissTypes=StaticMesh+Texture will simulate a miss on every access to to static mesh and texture keys.
-DDC-<Node>-MissTypes=StaticMesh@12.5+Texture will simulate misses on 12.5% of accesses to static mesh keys and every access to texture keys.
-DDC-<Node>-MissRate=5 -DDC-<Node>-MissTypes=Texture will simulate misses on every access to texture keys and 5% of other keys.
-DDC-<Node>-MissRate=5 will simulate misses on 5% of keys.
-DDC-<Node>-MissSalt=PositiveInt32 will set the salt used to match keys to simulate misses on. A key always has the same simulated miss behavior with the same salt.

#preflight 6244d102b33098a72dc136df
#rb Zousar.Shaker

[CL 19572200 by Devin Doucette in ue5-main branch]
2022-03-31 10:06:47 -04:00
Zousar Shaker
4b6d3f3a0a Fix bug where the Content-Type header had multiple conflicting entries when posting compact binary during Horde Storage communications:
+		[1]	L"Content-Type: application/x-ue-cb"	FString
+		[2]	L"Content-Type: application/x-www-form-urlencoded"	FString

#rb devin.doucette
#rnx
#preflight 623bfe051302f69e9a9b5e1e

[CL 19492306 by Zousar Shaker in ue5-main branch]
2022-03-24 01:28:28 -04:00
Zousar Shaker
03207d24ea Allow OAuth access scope to be configurable for a Horde Storage DDC store. Supplied by partners.
#rb devin.doucette
#rnx
#preflight 623b9723c73745c600c264e6

[CL 19487258 by Zousar Shaker in ue5-main branch]
2022-03-23 18:05:53 -04:00
steve robb
87fb605867 TCHAR array init fixes for UTF-8 mode.
#rb devin.doucette
#preflight 6231e536e2541b4ff3af6cd2

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19405353 via CL 19408955 via CL 19419621 via CL 19419706
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)

[CL 19420584 by steve robb in ue5-main branch]
2022-03-17 10:29:53 -04:00
zousar shaker
a33be10d6b Fix issue (using server-side feature in Horde Storage) with inconsistency to DDC content from Horde Storage.
This would manifest as a cache entry being deemed to "exist", but later when you attempt to fetch it, it would not exist.

#rb devin.doucette
[FYI] joakim.lindqvist
#jira UE-145701
#lockdown simon.tourangeau
#preflight 6231f8866e25767a2189e99a

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 19407779 in //UE5/Release-5.0/... via CL 19408966
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19409941 by zousar shaker in ue5-main branch]
2022-03-16 15:13:23 -04:00