Commit Graph

215 Commits

Author SHA1 Message Date
Devin Doucette
89eb2eef0f DDC: Fixed a compile error inside a checkfSlow
#rb trivial
#rnx

[CL 15830267 by Devin Doucette in ue5-main branch]
2021-03-25 15:52:40 -04:00
Devin Doucette
d87eb29d07 DDC: Switched to WriteToString in the Jupiter backend
#rb Zousar.Shaker
#rnx

[CL 15829627 by Devin Doucette in ue5-main branch]
2021-03-25 15:34:31 -04:00
Zousar Shaker
0ee734bf61 Fix HTTP/Jupiter DDC backend to account for the fact that Jupiter will return batched results in arbitrary order, not the same order that the requests/keys were listed.
#rb devin.doucette

[CL 15771001 by Zousar Shaker in ue5-main branch]
2021-03-22 18:23:46 -04:00
nick darnell
832cc08a58 DDC - Don't look up the store for DDC paths that don't have environment variable overrides.
#jira UE-110702

#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: nick.darnell
#ROBOMERGE-COMMAND: _robomerge UE5-Main
#ROBOMERGE-SOURCE: CL 15715366 via CL 15715370 via CL 15715374 via CL 15715407 via CL 15743064
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v781-15675533)

[CL 15747819 by nick darnell in ue5-main branch]
2021-03-19 11:39:18 -04:00
Marc Audy
0cbbc781ca Merge UE5/Release-Engine-Staging @ 15740152 to UE5/Main
This represents UE4/Main @ 15709114

[CL 15740605 by Marc Audy in ue5-main branch]
2021-03-18 15:20:03 -04:00
Joakim Lindqvist
5957a9c659 Added option to override Jupiter speed class
#rb devin.doucette

[CL 15736124 by Joakim Lindqvist in ue5-main branch]
2021-03-18 05:51:42 -04:00
Johan Torp
7b4d8d2785 Enable DDC1 Jupiter backend GET/PUT pooling and added dedicated main thread pools
#rb devin.doucette, johan.berg
#fyi jeff.farris
#jira none
#rnx

[CL 15718578 by Johan Torp in ue5-main branch]
2021-03-17 04:27:28 -04:00
Joakim Lindqvist
911fb5c9ab Send DDC payloads with IoHash instead of SHA1
#rb devin.doucette

[CL 15710022 by Joakim Lindqvist in ue5-main branch]
2021-03-16 06:11:58 -04:00
Devin Doucette
9a7c5e5a52 DDC: Fixed a static analysis warning in the cache bucket constructor
#rb trivial
#rnx

[CL 15697979 by Devin Doucette in ue5-main branch]
2021-03-15 13:05:50 -04:00
Devin Doucette
2da0cc5c74 DDC: Assert that modifications made by PutCachedData will not exceed 2 GiB
#rb Zousar.Shaker
#rnx

[CL 15681767 by Devin Doucette in ue5-main branch]
2021-03-11 18:12:21 -04:00
Devin Doucette
490d684d40 DDC: Clean up a few details in the new Cache API
#rb Zousar.Shaker
#rnx

[CL 15681766 by Devin Doucette in ue5-main branch]
2021-03-11 18:12:08 -04:00
Devin Doucette
7d5d9f3798 DDC: Clean up a few details in the new Cache API and remove the last exported type from DDC
#rb Zousar.Shaker
#rnx

[CL 15670275 by Devin Doucette in ue5-main branch]
2021-03-10 17:33:38 -04:00
Devin Doucette
017b41eb57 DDC: Refactored the new Cache API to support its use through only an interface dependency
#rb Zousar.Shaker
#rnx

[CL 15666872 by Devin Doucette in ue5-main branch]
2021-03-10 12:23:29 -04:00
Johan Berg
cfc09f48f2 Batched Jupiter queries
Adds a helper class to make CachedDataProbablyExists, GetCachedData and PutCachedData operations. The helper checks the availble connections in the pool and decides if the thread should make a regular request or batch together with other requests. This will reduce the number of concurrent connections on each client.

Disabled by default.

#rb devin.doucette

[CL 15604924 by Johan Berg in ue5-main branch]
2021-03-04 09:35:03 -04:00
Devin Doucette
2440bc5fbd DDC: Add generic EStatus and replace ECacheStatus with it
#rb Zousar.Shaker
#rnx

[CL 15583574 by Devin Doucette in ue5-main branch]
2021-03-03 11:55:38 -04:00
Joakim Lindqvist
22cf9dc2f7 Reduced Error output to Display in HttpBackend as this is a recoverable error.
#rb devin.doucette
#jira UE-109707

[CL 15580171 by Joakim Lindqvist in ue5-main branch]
2021-03-03 05:05:34 -04:00
devin doucette
0f5201b8a1 DDC: Added CreateOrGetDDC to the module and changed creation of DDC to occur there instead of on module startup
#rb Arciel.Rekman, Zousar.Shaker

#ROBOMERGE-OWNER: devin.doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 15530385 via CL 15530408 via CL 15530432 via CL 15530444 via CL 15531592
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v771-15082668)
#ROBOMERGE[STARSHIP]: Release-5.0-EarlyAccess

[CL 15531598 by devin doucette in ue5-main branch]
2021-02-25 14:50:04 -04:00
Joakim Lindqvist
c159fac11a Adding command line override for Jupiter Host in ShooterGame, DDCHttpHost
Changed the Dev toggle for CookCompare to use this.
This can also be used to locally toggle Jupiter to run against a local deployment.

[CL 15527157 by Joakim Lindqvist in ue5-main branch]
2021-02-25 06:12:30 -04:00
Devin Doucette
f255d91cdd DDC: Simplified comparison of FCacheBucket and the keys that depend on it
#rb Zousar.Shaker
#rnx

[CL 15514773 by Devin Doucette in ue5-main branch]
2021-02-24 11:46:41 -04:00
Devin Doucette
7e2e44ebf4 DDC: Refactored the Cache API to prepare for the Build API
- Removed the "Cache" prefix from ECachePriority, FCacheRequest, ICacheRequest, FCachePayload, FCachePayloadId to allow reuse in the Build API.
- Added FCompressedBuffer as the starting point for the API for creating and accessing compressed buffers.

#rb Zousar.Shaker
#rnx

[CL 15511723 by Devin Doucette in ue5-main branch]
2021-02-24 01:37:45 -04:00
Devin Doucette
7c5e0f99fa DDC: Capture cache value arrays into shared buffers instead of copying them
#rb Zousar.Shaker
#rnx

[CL 15501496 by Devin Doucette in ue5-main branch]
2021-02-23 12:32:38 -04:00
Devin Doucette
b5ae234c7c CompactBinary: Renamed view types to use a View suffix, and remove the Ref suffix from types that hold a SharedBuffer
#rb Zousar.Shaker
#rnx

[CL 15493834 by Devin Doucette in ue5-main branch]
2021-02-22 16:34:05 -04:00
Joakim Lindqvist
385e81f304 HttpDerivedBackend - Allow IoHash header as well
* Removed pointless double hashing for sha1 hashes.
* Fixed spelling recieved -> received

#rb devin.doucette

[CL 15489593 by Joakim Lindqvist in ue5-main branch]
2021-02-22 11:11:09 -04:00
Ben Marsh
4f78759ae2 Disable intermittent DDC warning when running on build machines.
#rb none
#rnx

[CL 15478823 by Ben Marsh in ue5-main branch]
2021-02-19 18:09:28 -04:00
Devin Doucette
c13b0dbecf DDC: Updated the new cache interface to expose compression of cache records
- FCacheRecord must now be created by FCacheRecordBuilder.
- FCacheRecord values are always an opaque buffer now, and ECacheRecordType has been removed.
- FCachePayload has been added as a common type for the cache record value and attachments.
- FCachePayloadId has been added as a 12-byte identifier for cache payloads.
- ICache::GetAttachments is now GetPayloads and can fetch both values and attachments.
- ECachePriority::Blocking has been added for requests that the caller will block on.

#rb Zousar.Shaker
#rnx

[CL 15471951 by Devin Doucette in ue5-main branch]
2021-02-19 11:26:34 -04:00