Commit Graph

360 Commits

Author SHA1 Message Date
devin doucette
55ac73b651 DDC: Renamed Dispatch[Op] to Schedule[Op] and tweaked formatting for consistency
#rb none
#rnx
#preflight 613a336abf5a710001674089

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17474273 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17474278 by devin doucette in ue5-release-engine-test branch]
2021-09-09 13:36:48 -04:00
johan torp
4afc055300 Add DDC2 memory scheduler to limit peak memory consumption
Also:
* Implemented rough memory usage estimate for texture cooking
* Fixed build action -> constant copying bug

#rb devin.doucette

#ROBOMERGE-AUTHOR: johan.torp
#ROBOMERGE-SOURCE: CL 17470303 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17470308 by johan torp in ue5-release-engine-test branch]
2021-09-09 06:04:13 -04:00
zousar shaker
8c80e5a866 Change Compact Binary Package API to represent binary attachments as compressed buffers identified by their raw hash.
Change Compact Binary Package serialization for binary attachments, compressed binary attachments and objects.

#rb devin.doucette

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 17450533 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17450587 by zousar shaker in ue5-release-engine-test branch]
2021-09-07 18:07:00 -04:00
aurel cordonnier
7f517562d5 Merge from Release-Engine-Staging @ 17438845 to Release-Engine-Test
This represents UE4/Main @17430120 and Dev-PerfTest @17437669

[CL 17439044 by aurel cordonnier in ue5-release-engine-test branch]
2021-09-06 12:23:53 -04:00
devin doucette
c39e30cfd7 Added missing includes of DerivedDataPayload.h found by the non-unity build
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17426590 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17426595 by devin doucette in ue5-release-engine-test branch]
2021-09-03 17:11:57 -04:00
devin doucette
9c01e2fc04 DDC: Moved FPayloadId into a dedicated header
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17426140 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17426194 by devin doucette in ue5-release-engine-test branch]
2021-09-03 16:30:53 -04:00
devin doucette
3d262ca7f0 CompressedBuffer: Removed the FormatName functionality and update its last remaining callers
#rb Paul.Chipchase, Zousar.Shaker
#rnx
#preflight 612feb940380e700016fc8e5

#ROBOMERGE-SOURCE: CL 17403021 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17403035 by devin doucette in ue5-release-engine-test branch]
2021-09-02 10:30:25 -04:00
matt peters
30ac4d4b22 Fix error in ZenDerivedDataBackend::Put. It needs to make payloads with missing data references to existing data in the CacheStore, and reject the put if any don't exist. This error was causing population of editordomain packages in a Zen store from a shareddatacache to fail.
Change GetZenData to only take TArray64, and to use PerformBlockingHead instead of PerformBlockingDownload if outputdata is null.
#rb Devin.Doucette
#rnx
[FYI] Stefan.Boberg

#ROBOMERGE-SOURCE: CL 17388568 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17388571 by matt peters in ue5-release-engine-test branch]
2021-09-01 12:26:46 -04:00
devin doucette
71a00b6367 DDC: Changed the heuristic for local cache backends to identify them more accurately
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17358471 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17358493 by devin doucette in ue5-release-engine-test branch]
2021-08-30 16:38:46 -04:00
devin doucette
ce01b1fb44 DDC: Stopped loading build inputs for local execution when execution is disabled
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17358468 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17358490 by devin doucette in ue5-release-engine-test branch]
2021-08-30 16:38:36 -04:00
devin doucette
a066ec0c49 DDC: Implemented existing cache debug options for the new cache stores
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17308400 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v861-17282326)

[CL 17308416 by devin doucette in ue5-release-engine-test branch]
2021-08-25 16:08:33 -04:00
devin doucette
b575156c6f DDC: Added request barriers to the build job constructors
The barriers are needed to handle re-entrant AdvanceToState calls within the constructor of jobs with Blocking priority.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17302241 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v861-17282326)

[CL 17302256 by devin doucette in ue5-release-engine-test branch]
2021-08-25 10:15:39 -04:00
devin doucette
275765055a DDC: Fixed build callback status when the policy allows querying the cache but not building
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17291662 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v861-17282326)

[CL 17291681 by devin doucette in ue5-release-engine-test branch]
2021-08-24 15:25:59 -04:00
devin doucette
ae7825b29c DDC: Added EBuildStatus::CacheKey and return EStatus::Ok from a build when no build or cache was requested
#rb Zousar.Shaker
#rnx
#preflight 611f304c008be900010bf006

#ROBOMERGE-SOURCE: CL 17247202 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17247215 by devin doucette in ue5-release-engine-test branch]
2021-08-20 10:04:41 -04:00
stefan boberg
85a811cb21 Refactored the UE-side Zen project store interface so that we now use a common HTTP interface implementation across cache / project store classes. No functional impact, tested with ShooterGame
#rb none

#ROBOMERGE-SOURCE: CL 17234463 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17234464 by stefan boberg in ue5-release-engine-test branch]
2021-08-19 11:13:10 -04:00
stefan boberg
e549b0b12d Updated Zen HTTP interfaces to accept large payloads as a preparation for unifying the cache/store client implementations
#ROBOMERGE-SOURCE: CL 17218879 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17218897 by stefan boberg in ue5-release-engine-test branch]
2021-08-18 14:12:33 -04:00
stefan boberg
2d5b5e3409 DLL-exported functions for Zen HTTP interfaces to allow them to be consumed from other modules, and renamed them to make it easier to search for them (previous names were derived from the Jupiter HTTP back-end)
#rb trivial

#ROBOMERGE-SOURCE: CL 17187385 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17187388 by stefan boberg in ue5-release-engine-test branch]
2021-08-17 03:32:28 -04:00
andriy tylychko
6bf3101dcd deprecated FTicker and family and replaced by thread-safe FTSTicker
#jira UE-120090
#rb francis.hurteau


#ROBOMERGE-SOURCE: CL 17176325 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17176374 by andriy tylychko in ue5-release-engine-test branch]
2021-08-16 11:09:22 -04:00
devin doucette
d5a632673b DDC: Fixed a build job assert caused by canceling at the wrong time
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17108375 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17108399 by devin doucette in ue5-release-engine-test branch]
2021-08-09 17:56:38 -04:00
charles bloom
b9f505a152 Log DerivedDataCache when registry overrides dirs
make editor dialog update when string is empty

#rb devin.doucette

#ROBOMERGE-SOURCE: CL 17107690 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17107751 by charles bloom in ue5-release-engine-test branch]
2021-08-09 17:06:13 -04:00
devin doucette
d55593526e DDC: Add FBuildConfigContext::SetRequiredMemory() and forward the estimate to the build scheduler
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17088443 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17089009 by devin doucette in ue5-release-engine-test branch]
2021-08-06 17:53:47 -04:00
devin doucette
af0f2d277f DDC: Added flags to control request barrier behavior
#rb Matt.Peters
#rnx

#ROBOMERGE-SOURCE: CL 17088062 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17088067 by devin doucette in ue5-release-engine-test branch]
2021-08-06 17:11:37 -04:00
devin doucette
c67fcf1702 DDC: Split DerivedDataRequest.h and rename DerivedDataBuildPolicy.h
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17086960 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17086977 by devin doucette in ue5-release-engine-test branch]
2021-08-06 15:50:00 -04:00
devin doucette
bd60f3ef1b DDC: Removed ICacheFactory in favor of exported functions
FCacheBucket is now always exposed as FAnsiStringView.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17084521 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17084549 by devin doucette in ue5-release-engine-test branch]
2021-08-06 12:53:08 -04:00
devin doucette
7ae4b64cd6 DDC: Removed IBuild::Load[Type] in favor of exported functions
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17083958 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17083970 by devin doucette in ue5-release-engine-test branch]
2021-08-06 11:58:24 -04:00