Commit Graph

23 Commits

Author SHA1 Message Date
devin doucette
e513641c09 DDC: Check for existence of inputs in the build worker
Cancel unfinished builds when an error occurs that will lead to a non-zero exit code, and fix handling of -I/-Input.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17210835 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17210851 by devin doucette in ue5-release-engine-test branch]
2021-08-17 23:45:23 -04:00
zousar shaker
99d69ed028 Avoid parsing '+' characters as separators of a multi-value commandline argument for DerivedDataBuildWorkers as it causes bugs when a simple path has a '+' character in it as occurs on the build farm.
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17093365 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17093370 by zousar shaker in ue5-release-engine-test branch]
2021-08-07 01:32:39 -04:00
devin doucette
af0f2d277f DDC: Added flags to control request barrier behavior
#rb Matt.Peters
#rnx

#ROBOMERGE-SOURCE: CL 17088062 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17088067 by devin doucette in ue5-release-engine-test branch]
2021-08-06 17:11:37 -04:00
devin doucette
c67fcf1702 DDC: Split DerivedDataRequest.h and rename DerivedDataBuildPolicy.h
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17086960 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17086977 by devin doucette in ue5-release-engine-test branch]
2021-08-06 15:50:00 -04:00
devin doucette
7ae4b64cd6 DDC: Removed IBuild::Load[Type] in favor of exported functions
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17083958 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17083970 by devin doucette in ue5-release-engine-test branch]
2021-08-06 11:58:24 -04:00
devin doucette
f9a760f21f DDC: Rename FRequestGroup to FRequestOwner and export its constructor
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17074555 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17074567 by devin doucette in ue5-release-engine-test branch]
2021-08-05 15:46:22 -04:00
devin doucette
c1eddbc7ec DDC: Fixed a compile error in the build worker
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 17071429 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17071446 by devin doucette in ue5-release-engine-test branch]
2021-08-05 13:16:50 -04:00
devin doucette
bfda02d384 DDC: Replaced FRequest with FRequestGroup to handle nested and chained requests more robustly
Any function that may create a request now has a IRequestOwner& parameter, and uses the Begin and End functions on the owner to manage the lifetime of any requests that it creates, as well as using End to invoke the completion callback for any request which has one.

The new FRequestBarrier may be used to block a group from being considered complete in a scope where more requests may be added to it.

#rb Matt.Peters
#rnx
#preflight 6109b5c403d303000144cce5
#preflight 610acf7103d30300016fda94

#ROBOMERGE-SOURCE: CL 17060470 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v850-17047176)

[CL 17060649 by devin doucette in ue5-release-engine-test branch]
2021-08-04 18:08:50 -04:00
zousar shaker
b8bb77e28d DDC2 remote execution cleanup:
-use newer FCompressedBuffer::Compress method when providing buffers for worker executable files
-use the preferred *.action and *.output file extensions over *.uddba and *.uddbo
-use the preferred "-Build=" commandline argument over "-b="

#rb devin.doucette

#ROBOMERGE-SOURCE: CL 16938228 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 16938260 by zousar shaker in ue5-release-engine-test branch]
2021-07-23 12:17:08 -04:00
zousar shaker
acc47afb6d Add build worker discovery mechanism using target receipt files.
#rb matt.peters

#ROBOMERGE-SOURCE: CL 16917716 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16917884 by zousar shaker in ue5-release-engine-test branch]
2021-07-21 19:12:13 -04:00
zousar shaker
e4289118dc Force load of all linked modules during initialization of a standalone build worker process.
#rb devin.doucette
#preflight 60f6ddab915a9200018a83c4

#ROBOMERGE-SOURCE: CL 16897796 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16897798 by zousar shaker in ue5-release-engine-test branch]
2021-07-20 12:11:04 -04:00
zousar shaker
08a024596c Fix linux compile error and bug with my last submits related to function version iteration for build functions.
#rb none

#ROBOMERGE-SOURCE: CL 16889472 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16889487 by zousar shaker in ue5-release-engine-test branch]
2021-07-19 17:41:29 -04:00
devin doucette
58774329d7 DDC: Exposed the cache key and build status in the build complete callback params
#rb Zousar.Shaker
#rnx
#preflight 60f5c7a78da1560001c68086

#ROBOMERGE-SOURCE: CL 16888763 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16888787 by devin doucette in ue5-release-engine-test branch]
2021-07-19 16:49:25 -04:00
zousar shaker
aa6a94b410 Allow build workers to report the function names and versions they contain.
#rb devin.doucette

#ROBOMERGE-SOURCE: CL 16888229 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16888249 by zousar shaker in ue5-release-engine-test branch]
2021-07-19 16:16:56 -04:00
zousar shaker
a48d9e9497 Expand texture build functions to all current texture formats.
Expand texture build workers to all current platform specific texture formats, or added build functions to the base build worker.  Workers are buildable, but not discoverable yet as discovery will be refactored soon to use Target Receipts.
Reduce boilerplate involved in setup of build worker.

#rb devin.doucette

#ROBOMERGE-SOURCE: CL 16853856 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16853877 by zousar shaker in ue5-release-engine-test branch]
2021-07-14 15:27:36 -04:00
devin doucette
446d078cfd DDC: Fixed the formatting of an error message
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 16713229 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16713283 by devin doucette in ue5-release-engine-test branch]
2021-06-17 17:55:08 -04:00
Devin Doucette
17aa2edfd1 DDC: Added a null check for GWarn in the build worker in case logging is disabled
#rb trivial
#rnx

[CL 16538465 by Devin Doucette in ue5-main branch]
2021-06-02 15:58:33 -04:00
Devin Doucette
74f85e31f7 DDC: Convert the build worker to use the build session and input resolver
#rb Zousar.Shaker
#rnx

[CL 16514567 by Devin Doucette in ue5-main branch]
2021-06-01 11:18:33 -04:00
Zousar Shaker
9c5d291d34 Fix DerivedDataBuildLoop.cpp
#rb trivial

[CL 16513126 by Zousar Shaker in ue5-main branch]
2021-06-01 09:05:28 -04:00
Zousar Shaker
18aa6a10e5 Ensure the build workers don't use a DDC backend graph.
#rb trivial

[CL 16455377 by Zousar Shaker in ue5-main branch]
2021-05-25 16:17:21 -04:00
Zousar Shaker
a6bd40e38d -Fix bug in the interpretation of mip offset in FTextureBuildFunction (it is offset into uncomrpessed data, not offset into compressed data)
-Added more metadata to the output of the texture export feature.
-Make FTextureBuildFunction stateless and abstract.  Now must be derived to produce an instantiable build function.
-Made PS5TextureFormat, TextureFormatOodle, and TextureFormatUncompressed modules depend on TextureBuild module and implement concrete derivatives of FTextureBuildFunction.
-Made PS5TextureFormat, TextureFormatOodle, and TextureFormatUncompressed modules register their build functions through an IBuildFunctionFactory for the lifetime of the module.
-Removed all public interface for the DerivedDataBuild{Loop, Worker} and have them just use the IBuildFunctionFactory modular feature to find all linked build functions.

#rb devin.doucette
#preflight 60acfb486905a60001c3cd29

[CL 16448407 by Zousar Shaker in ue5-main branch]
2021-05-25 10:37:34 -04:00
Zousar Shaker
bc073c41ed Change DerivedDataBuildWorker/Loop & TextureDerivedDataBuildExporter to use the DDC implementation of FBuildAction[Builder] and FBuildOutput[Builder]
#rb devin.doucette
#preflight 60a7e6ad7370d30001cd62cc

[CL 16421772 by Zousar Shaker in ue5-main branch]
2021-05-21 14:18:04 -04:00
Zousar Shaker
5234e8fb12 Define first TextureBuildWorkers for use with DDC as well as a DDC build worker base module used as an entrypoint for standalone worker binaries.
#rb matt.peters

[CL 16323208 by Zousar Shaker in ue5-main branch]
2021-05-13 20:42:14 -04:00