Commit Graph

430 Commits

Author SHA1 Message Date
devin doucette
6f080fc3f1 DDC: Added a ServerID field to the Jupiter configuration to allow reading settings from another section
When ServerID is used, configuration is first loaded from a key matching the ServerID field, in the HordeStorageServers section of the same INI. This makes it easier to put Jupiter configuration in a consistent place and have it stripped by the cooker to avoid leaking any of the configuration.

#rb Zousar.Shaker
#rnx
#preflight 617a28570cec4300011619d4

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17961730 in //UE5/Release-5.0/... via CL 17961740
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v885-17909292)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17961744 by devin doucette in ue5-release-engine-test branch]
2021-10-28 09:39:12 -04:00
devin doucette
cf9f616446 DDC: Fixed a static analysis warning in the memory backend
#jira UE-132800
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17926730 in //UE5/Release-5.0/... via CL 17927106
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v885-17909292)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17927130 by devin doucette in ue5-release-engine-test branch]
2021-10-26 12:33:40 -04:00
joakim lindqvist
eec8f48f1a Support reading HttpBackend secret from a file
#ROBOMERGE-AUTHOR: joakim.lindqvist
#ROBOMERGE-SOURCE: CL 17921575 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v885-17909292)

[CL 17922647 by joakim lindqvist in ue5-release-engine-test branch]
2021-10-26 09:14:51 -04:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
devin doucette
dc4658fa91 DDC: Updated IBuildWorkerExecutor to allow filtering of payloads
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17876434 in //UE5/Release-5.0/... via CL 17876436
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17876451 by devin doucette in ue5-release-engine-test branch]
2021-10-20 14:50:22 -04:00
devin doucette
01c33e6702 DDC: Reworked FBuildSession to allow filtering of payloads
- Build now takes optional inputs for consistency with BuildAction.
- BuildAction has been renamed to Build and works the same as Build except it starts from an action.
- Build now takes a FBuildPolicy, which is implicitly constructible from EBuildPolicy, and allows setting the policy by payload.
- FBuildContext and FBuildConfigContext now have policy masks that allow clearing flags across every payload in the build.
- EBuildPolicy has CacheStore split into CacheStoreOnQuery and CacheStoreOnBuild for more control of cache behavior.
- Builds now use consistent completion callback parameters, FBuildCompleteParams.
- Build types now have debugger visualization.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17874670 in //UE5/Release-5.0/... via CL 17874724
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17874740 by devin doucette in ue5-release-engine-test branch]
2021-10-20 12:38:01 -04:00
devin doucette
10df2328dd DDC: Added structured cache support to the throttle wrapper
#rb Zousar.Shaker
#rnx
[FYI] Danny.Couture

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17872207 in //UE5/Release-5.0/... via CL 17872235
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17874113 by devin doucette in ue5-release-engine-test branch]
2021-10-20 11:50:36 -04:00
devin doucette
9cbcc4508a DDC: Limit async put priority to Highest to avoid blocking
#rb Matt.Peters
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17866436 in //UE5/Release-5.0/... via CL 17866457
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17866461 by devin doucette in ue5-release-engine-test branch]
2021-10-19 17:31:09 -04:00
devin doucette
4025bebb2d DDC: Fixed fetching of partial records in the filesystem backend
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17860656 in //UE5/Release-5.0/... via CL 17860683
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17860687 by devin doucette in ue5-release-engine-test branch]
2021-10-19 10:38:14 -04:00
devin doucette
716e45a49c DDC: Reset the cache record builder in Build()
#rb trivial
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17859877 in //UE5/Release-5.0/... via CL 17859906
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17859915 by devin doucette in ue5-release-engine-test branch]
2021-10-19 09:41:36 -04:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00
devin doucette
3331f68391 DDC: Added ECachePolicy::PartialOnError and implemented partial cache records in the filesystem and memory backends
#rb Zousar.Shaker
#rnx
#preflight 615f703dcec44a000146cf80

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17759092 in //UE5/Release-5.0/... via CL 17760688
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v879-17706426)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17760906 by devin doucette in ue5-release-engine-test branch]
2021-10-08 11:01:21 -04:00
Devin Doucette
a077feffbd DDC: Reworked ICacheStore to allow partial records, filtering of payloads, and loading parts of payloads
- ICacheStore::Put() has updated documentation to reflect the requirements of partial records.
- ICacheStore::Get() now takes a FCacheRecordPolicy, which is implicitly constructible from ECachePolicy, and allows setting the policy by payload.
- ICacheStore::GetPayload() is replaced by ICacheStore::GetChunks(), which allows loading parts of payloads.
- ICacheStore::CancelAll() is moved to ICache::CancelAll() because the cache can track requests at the top level and cancel them without exposing cancellation on individual cache stores.
- ECachePolicy::SkipLocalCopy has been removed because it is difficult to reason about.
- ECachePolicy::SkipData flags now have a documented meaning for put requests, to hint that record existence implies payload existence.
- The filesystem and memory cache stores have been updated to support partial records, filtering of payloads, and loading parts of payloads.
- Requesting part of a payload will decompress the entire payload for now, until compressed buffers expose a way to decompress only part.
- Fixed a bug in FTexturePlatformData::AreDerivedMipsAvailable() that caused it to return false for structured cache keys.

#rb Zousar.Shaker
#rnx
#preflight 615e03241ed62f0001b95454

#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17748550 in //UE5/Release-5.0/... via CL 17748555
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v879-17706426)
#ROBOMERGE-CONFLICT from-shelf
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17748602 by Devin Doucette in ue5-release-engine-test branch]
2021-10-07 09:11:32 -04:00
ionut matasaru
2f3404b10e Fixed usage of UE_LOG (arguments for "%s" formatting).
#rb Zousar.Shaker

#ROBOMERGE-AUTHOR: ionut.matasaru
#ROBOMERGE-SOURCE: CL 17691626 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v875-17642767)
#ROBOMERGE[STARSHIP]: UE5-Release-Engine-Staging Release-5.0

[CL 17691646 by ionut matasaru in ue5-release-engine-test branch]
2021-10-01 10:30:26 -04:00
devin doucette
06b8129003 DDC: Removed DDCUtils in favor of a config-based approach that can work for any project
#rb Tim.Smith
#preflight 6154951003f33200010389a5

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17664358 in //UE5/Release-5.0/... via CL 17664461
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v875-17642767)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17664511 by devin doucette in ue5-release-engine-test branch]
2021-09-29 13:47:28 -04:00
per larsson
fa26f9c012 Make content-type/accept headers mandatory
#rb Stefan.Boberg
#rnx

#ROBOMERGE-AUTHOR: per.larsson
#ROBOMERGE-SOURCE: CL 17645289 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v875-17642767)

[CL 17645303 by per larsson in ue5-release-engine-test branch]
2021-09-28 09:53:06 -04:00
devin doucette
e5afff71d4 DDC: Switched to FLazyEvent to reduce build execution overhead
#rb Zousar.Shaker
#rnx
#preflight 614d5c567684f50001a07c40

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17629838 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17629859 by devin doucette in ue5-release-engine-test branch]
2021-09-27 09:36:23 -04:00
stefan boberg
9a31aa9363 Completed FZenHttpRequest::GetResponseAsPackage() implementation, matches Zen side CbPackage serialization
#rb none
#rnx

#ROBOMERGE-AUTHOR: stefan.boberg
#ROBOMERGE-SOURCE: CL 17621919 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17621929 by stefan boberg in ue5-release-engine-test branch]
2021-09-24 12:34:41 -04:00
zousar shaker
4da5e948f4 Fix missing ZenDerivedDataBackend stats.
#rb trivial
[FYI] devin.doucette
#preflight 614c0c0203e28a00011fc1d3

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 17604816 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17604821 by zousar shaker in ue5-release-engine-test branch]
2021-09-23 02:00:09 -04:00
zousar shaker
96f8eb0eca Avoid a compile error due to narrowing of types when compiling win64 editor with clang.
#rb trivial

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 17603577 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17603598 by zousar shaker in ue5-release-engine-test branch]
2021-09-22 22:02:15 -04:00
zousar shaker
96d0784963 Allow ZenServer to be autolaunched by editor use cases that require its presence without insisting that it always be autolaunched.
#rb matt.peters
#preflight 614b4150ee16e20001b146e3

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 17597162 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17597204 by zousar shaker in ue5-release-engine-test branch]
2021-09-22 13:00:17 -04:00
devin doucette
cf19925e08 DDC: Made CacheRecord serialized field order consistent with other types
#rb none
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17583018 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17583024 by devin doucette in ue5-release-engine-test branch]
2021-09-21 11:31:48 -04:00
per larsson
c6fafd4b7a DDC2: Use compressed buffer as attachment
#rb Devin.Doucette
#rnx

#ROBOMERGE-AUTHOR: per.larsson
#ROBOMERGE-SOURCE: CL 17580737 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17580740 by per larsson in ue5-release-engine-test branch]
2021-09-21 04:51:59 -04:00
devin doucette
750419503e DDC: Added ECachePolicy::KeepAlive and EBuildPolicy::CacheKeepAlive
The new flags are meant to prevent a cache store from evicting the associated cache records for the duration of the session.

#rb Zousar.Shaker
#rnx
#preflight 614954b611925a00017c525f

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17579887 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17579899 by devin doucette in ue5-release-engine-test branch]
2021-09-21 01:10:55 -04:00
stefan boberg
553dd501f1 Package marshaling now properly flags the root object as compressed
#rb none
#rnx

#ROBOMERGE-AUTHOR: stefan.boberg
#ROBOMERGE-SOURCE: CL 17573969 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17573973 by stefan boberg in ue5-release-engine-test branch]
2021-09-20 15:03:03 -04:00