35 Commits

Author SHA1 Message Date
devin doucette
01cb852a23 Deprecated the old names and paths for TSharedString
#rb Zousar.Shaker
#rnx

[CL 34567121 by devin doucette in ue5-main branch]
2024-06-21 12:52:32 -04:00
Devin Doucette
27dab83354 DDC: Follow-up to allowing the build system to be created without a cache
This change was included when testing but missing from the submitted change.

#preflight 637ff6f9f514e1ded9e55ea7
#rb none

[CL 23262867 by Devin Doucette in ue5-main branch]
2022-11-24 18:14:06 -05:00
Devin Doucette
e3f3ab918f DDC: Changed the build worker shutdown flow to more closely approximate the editor
#preflight 627c0dad0a5817c9d958962f
#rb Dan.Thompson
#rnx

[CL 20148457 by Devin Doucette in ue5-main branch]
2022-05-11 16:05:34 -04:00
Joe Kirchoff
ca7123d0dc DerivedDataBuildWorker: RequestEngineExit() when terminating
#rnx
#rb Zousar.Shaker
#preflight 623925118900c14eecb611a9

[CL 19462563 by Joe Kirchoff in ue5-main branch]
2022-03-21 21:50:20 -04:00
devin doucette
cdedb37d66 DDC: Converted the Build API to UTF-8 where appropriate
#jira UE-133382
#rb Zousar.Shaker
#rnx
#preflight 61e798ee3f00a0a23ef78c49

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18655307 in //UE5/Release-5.0/... via CL 18655320 via CL 18655322
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v900-18638592)

[CL 18655349 by devin doucette in ue5-main branch]
2022-01-19 00:27:48 -05:00
devin doucette
35393bbb2b DDC: Split FPayload into separate FValue and FValueId types
A payload was conceptually a value with an ID. That has been formalized by removing the ID from the payload and having separate FValue and FValueId types. This separation cleans up the API in a few areas, and provides a more natural path to providing a basic key/value API.

#rb Zousar.Shaker
#rnx
#preflight 61d704c04c252480ca284d61

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18531844 in //UE5/Release-5.0/... via CL 18531856
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18531864 by devin doucette in ue5-release-engine-test branch]
2022-01-06 11:05:57 -05:00
joe kirchoff
e736b5ae1a TextureBuildWorker: Force set -unattended when building
#rnx
#rb trivial

#ROBOMERGE-AUTHOR: joe.kirchoff
#ROBOMERGE-SOURCE: CL 18414309 in //UE5/Release-5.0/... via CL 18414310
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18414312 by joe kirchoff in ue5-release-engine-test branch]
2021-12-08 19:07:09 -05:00
devin doucette
90cc5a61d4 DDC: Fixed a compile error in the derived data build worker
#rb none
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18385446 in //UE5/Release-5.0/... via CL 18385452
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v896-18170469)

[CL 18385458 by devin doucette in ue5-release-engine-test branch]
2021-12-06 13:51:37 -05:00
devin doucette
27c1393427 CompressedBuffer: Removed partial decompression from FCompressedBuffer now that FCompressedBufferReader is available
Requiring the use of a separate reader type makes it more likely that readers will be reused, and makes it easier to audit reader usage going forward. Reusing readers is desirable to reduce the number of large temporary allocations made during partial decompression of a buffer.

- Added FCompressedBuffer::Save(FArchive&) and renamed FromCompressed(FArchive&) to Load(FArchive&).
- Added FCompressedBufferReaderSourceScope to set a buffer source within a scope.
- Added proper bounds checks to FNoneDecoder.
- Store the header checksum on the decoder context to allow raw blocks to be reused across sources.
- Decode the header on the fly to avoid a temporary header allocation when the header is in contiguous memory.

#rb Zousar.Shaker
#rnx
#preflight 61a98d53800738dbfbc84c73

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18382211 in //UE5/Release-5.0/... via CL 18382310
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v896-18170469)

[CL 18382377 by devin doucette in ue5-release-engine-test branch]
2021-12-06 10:16:05 -05:00
devin doucette
cea98e6ab4 DDC: Split build diagnostics into messages and logs
Messages are deterministic output recorded explicitly by the build function. Logs will be automatically captured from the output log and are not necessarily deterministic. The presence of logs in a build output automatically disables caching of the build output.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18286155 in //UE5/Release-5.0/... via CL 18286173
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18286201 by devin doucette in ue5-release-engine-test branch]
2021-11-24 13:25:25 -05:00
devin doucette
a10825eaa4 DDC: Updated DerivedDataBuildWorker to use the new build session interface
#rb trivial
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17875199 in //UE5/Release-5.0/... via CL 17875215
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17875226 by devin doucette in ue5-release-engine-test branch]
2021-10-20 13:10:43 -04:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00
devin doucette
e513641c09 DDC: Check for existence of inputs in the build worker
Cancel unfinished builds when an error occurs that will lead to a non-zero exit code, and fix handling of -I/-Input.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17210835 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17210851 by devin doucette in ue5-release-engine-test branch]
2021-08-17 23:45:23 -04:00
zousar shaker
99d69ed028 Avoid parsing '+' characters as separators of a multi-value commandline argument for DerivedDataBuildWorkers as it causes bugs when a simple path has a '+' character in it as occurs on the build farm.
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17093365 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17093370 by zousar shaker in ue5-release-engine-test branch]
2021-08-07 01:32:39 -04:00
devin doucette
af0f2d277f DDC: Added flags to control request barrier behavior
#rb Matt.Peters
#rnx

#ROBOMERGE-SOURCE: CL 17088062 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17088067 by devin doucette in ue5-release-engine-test branch]
2021-08-06 17:11:37 -04:00
devin doucette
c67fcf1702 DDC: Split DerivedDataRequest.h and rename DerivedDataBuildPolicy.h
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17086960 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17086977 by devin doucette in ue5-release-engine-test branch]
2021-08-06 15:50:00 -04:00
devin doucette
7ae4b64cd6 DDC: Removed IBuild::Load[Type] in favor of exported functions
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17083958 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17083970 by devin doucette in ue5-release-engine-test branch]
2021-08-06 11:58:24 -04:00
devin doucette
f9a760f21f DDC: Rename FRequestGroup to FRequestOwner and export its constructor
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17074555 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17074567 by devin doucette in ue5-release-engine-test branch]
2021-08-05 15:46:22 -04:00
devin doucette
c1eddbc7ec DDC: Fixed a compile error in the build worker
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 17071429 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17071446 by devin doucette in ue5-release-engine-test branch]
2021-08-05 13:16:50 -04:00
devin doucette
bfda02d384 DDC: Replaced FRequest with FRequestGroup to handle nested and chained requests more robustly
Any function that may create a request now has a IRequestOwner& parameter, and uses the Begin and End functions on the owner to manage the lifetime of any requests that it creates, as well as using End to invoke the completion callback for any request which has one.

The new FRequestBarrier may be used to block a group from being considered complete in a scope where more requests may be added to it.

#rb Matt.Peters
#rnx
#preflight 6109b5c403d303000144cce5
#preflight 610acf7103d30300016fda94

#ROBOMERGE-SOURCE: CL 17060470 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v850-17047176)

[CL 17060649 by devin doucette in ue5-release-engine-test branch]
2021-08-04 18:08:50 -04:00
zousar shaker
b8bb77e28d DDC2 remote execution cleanup:
-use newer FCompressedBuffer::Compress method when providing buffers for worker executable files
-use the preferred *.action and *.output file extensions over *.uddba and *.uddbo
-use the preferred "-Build=" commandline argument over "-b="

#rb devin.doucette

#ROBOMERGE-SOURCE: CL 16938228 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 16938260 by zousar shaker in ue5-release-engine-test branch]
2021-07-23 12:17:08 -04:00
zousar shaker
e4289118dc Force load of all linked modules during initialization of a standalone build worker process.
#rb devin.doucette
#preflight 60f6ddab915a9200018a83c4

#ROBOMERGE-SOURCE: CL 16897796 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16897798 by zousar shaker in ue5-release-engine-test branch]
2021-07-20 12:11:04 -04:00
zousar shaker
08a024596c Fix linux compile error and bug with my last submits related to function version iteration for build functions.
#rb none

#ROBOMERGE-SOURCE: CL 16889472 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16889487 by zousar shaker in ue5-release-engine-test branch]
2021-07-19 17:41:29 -04:00
devin doucette
58774329d7 DDC: Exposed the cache key and build status in the build complete callback params
#rb Zousar.Shaker
#rnx
#preflight 60f5c7a78da1560001c68086

#ROBOMERGE-SOURCE: CL 16888763 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16888787 by devin doucette in ue5-release-engine-test branch]
2021-07-19 16:49:25 -04:00
zousar shaker
aa6a94b410 Allow build workers to report the function names and versions they contain.
#rb devin.doucette

#ROBOMERGE-SOURCE: CL 16888229 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16888249 by zousar shaker in ue5-release-engine-test branch]
2021-07-19 16:16:56 -04:00