30 Commits

Author SHA1 Message Date
devin doucette
e675fcb5b5 DDC: Exposed metadata more extensively in the build API
#rb Zousar.Shaker

[CL 30691096 by devin doucette in ue5-main branch]
2024-01-18 14:06:50 -05:00
devin doucette
72f0249b95 Replaced uses of FLazyEvent with FManualResetEvent
#rb none
#rnx

[CL 24810985 by devin doucette in ue5-main branch]
2023-03-27 18:26:53 -04:00
devin doucette
f52cece901 Reverted the conversion of FLazyEvent to FManualResetEvent while investigating an issue
#rb none
#rnx

[CL 24763649 by devin doucette in ue5-main branch]
2023-03-23 12:35:09 -04:00
devin doucette
18dfce3eca Added FManualResetEvent
This is a one-byte manual-reset event that can replace that use case for the experimental FLazyEvent.

#rb Zousar.Shaker

[CL 24756514 by devin doucette in ue5-main branch]
2023-03-22 19:47:06 -04:00
Devin Doucette
6c33c94b36 DDC: Added experimental interface IBuildSchedulerThreadPoolProvider to integrate the build scheduler with the asset compiling managers
#jira UE-141137
#jira UE-141139
#preflight 63696f5e843e6ac79482dad1
#rb Zousar.Shaker

[CL 23027624 by Devin Doucette in ue5-main branch]
2022-11-08 09:45:50 -05:00
steve robb
fd2a0b1e20 Removal of TEXT literal concatenations, which doesn't work in UTF-8 mode.
#rb devin.doucette
#jira UE-132142
#preflight 62a8748fa76c84377748036b

#ROBOMERGE-OWNER: steve.robb
#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 20647443 via CL 20648283 via CL 20648315
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v955-20579017)

[CL 20651786 by steve robb in ue5-main branch]
2022-06-14 12:46:34 -04:00
Devin Doucette
92a888c1f6 Replaced uses of _ASV with ANSITEXTVIEW
#preflight 6266acf50634d0904cdfe91c
#rb Steve.Robb
#rnx

[CL 19903251 by Devin Doucette in ue5-main branch]
2022-04-25 11:31:36 -04:00
devin doucette
517947112b DDC: Reorganized cache policy code into its own files
- FCacheRecordPolicyBuilder now disallows adding value policies with flags that only make sense for records.
- FCacheRecordPolicyBuilder now disallows adding multiple value policies for the same value.
- FCacheRecordPolicyBuilder now skips addition of value policies that match the default policy.

#rb Matt.Peters
#rnx
#preflight 61f471cc6b5aea38e5c64cb6

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18782116 in //UE5/Release-5.0/... via CL 18782123 via CL 18782168
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18782178 by devin doucette in ue5-main branch]
2022-01-28 20:34:31 -05:00
devin doucette
cdedb37d66 DDC: Converted the Build API to UTF-8 where appropriate
#jira UE-133382
#rb Zousar.Shaker
#rnx
#preflight 61e798ee3f00a0a23ef78c49

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18655307 in //UE5/Release-5.0/... via CL 18655320 via CL 18655322
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v900-18638592)

[CL 18655349 by devin doucette in ue5-main branch]
2022-01-19 00:27:48 -05:00
devin doucette
23f9d839c7 DDC: Renamed ECachePolicy::PartialOnError to PartialRecord
The name more accurately reflects the purpose of the flag, which is to allow a partial record to be fetched or stored in the absence of data for otherwise-required values.

#rb Zousar.Shaker
#rnx
#preflight 61e5ae87a2616066f68abc35

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18634974 in //UE5/Release-5.0/... via CL 18634979 via CL 18634988
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18637552 by devin doucette in ue5-main branch]
2022-01-18 04:45:13 -05:00
devin doucette
35393bbb2b DDC: Split FPayload into separate FValue and FValueId types
A payload was conceptually a value with an ID. That has been formalized by removing the ID from the payload and having separate FValue and FValueId types. This separation cleans up the API in a few areas, and provides a more natural path to providing a basic key/value API.

#rb Zousar.Shaker
#rnx
#preflight 61d704c04c252480ca284d61

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18531844 in //UE5/Release-5.0/... via CL 18531856
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18531864 by devin doucette in ue5-release-engine-test branch]
2022-01-06 11:05:57 -05:00
devin doucette
8161902f38 DDC: Exposed payload compression block size through the cache record and build function interfaces
#rb Zousar.Shaker
#rnx
#preflight 61b7721bdd0e83c861a420d8

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18444327 in //UE5/Release-5.0/... via CL 18445345
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18445509 by devin doucette in ue5-release-engine-test branch]
2021-12-13 13:44:06 -05:00
devin doucette
cea98e6ab4 DDC: Split build diagnostics into messages and logs
Messages are deterministic output recorded explicitly by the build function. Logs will be automatically captured from the output log and are not necessarily deterministic. The presence of logs in a build output automatically disables caching of the build output.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18286155 in //UE5/Release-5.0/... via CL 18286173
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18286201 by devin doucette in ue5-release-engine-test branch]
2021-11-24 13:25:25 -05:00
devin doucette
01c33e6702 DDC: Reworked FBuildSession to allow filtering of payloads
- Build now takes optional inputs for consistency with BuildAction.
- BuildAction has been renamed to Build and works the same as Build except it starts from an action.
- Build now takes a FBuildPolicy, which is implicitly constructible from EBuildPolicy, and allows setting the policy by payload.
- FBuildContext and FBuildConfigContext now have policy masks that allow clearing flags across every payload in the build.
- EBuildPolicy has CacheStore split into CacheStoreOnQuery and CacheStoreOnBuild for more control of cache behavior.
- Builds now use consistent completion callback parameters, FBuildCompleteParams.
- Build types now have debugger visualization.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17874670 in //UE5/Release-5.0/... via CL 17874724
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17874740 by devin doucette in ue5-release-engine-test branch]
2021-10-20 12:38:01 -04:00
devin doucette
e5afff71d4 DDC: Switched to FLazyEvent to reduce build execution overhead
#rb Zousar.Shaker
#rnx
#preflight 614d5c567684f50001a07c40

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17629838 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17629859 by devin doucette in ue5-release-engine-test branch]
2021-09-27 09:36:23 -04:00
devin doucette
750419503e DDC: Added ECachePolicy::KeepAlive and EBuildPolicy::CacheKeepAlive
The new flags are meant to prevent a cache store from evicting the associated cache records for the duration of the session.

#rb Zousar.Shaker
#rnx
#preflight 614954b611925a00017c525f

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17579887 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17579899 by devin doucette in ue5-release-engine-test branch]
2021-09-21 01:10:55 -04:00
devin doucette
c39e30cfd7 Added missing includes of DerivedDataPayload.h found by the non-unity build
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17426590 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17426595 by devin doucette in ue5-release-engine-test branch]
2021-09-03 17:11:57 -04:00
devin doucette
c67fcf1702 DDC: Split DerivedDataRequest.h and rename DerivedDataBuildPolicy.h
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17086960 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17086977 by devin doucette in ue5-release-engine-test branch]
2021-08-06 15:50:00 -04:00
devin doucette
bd60f3ef1b DDC: Removed ICacheFactory in favor of exported functions
FCacheBucket is now always exposed as FAnsiStringView.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 17084521 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17084549 by devin doucette in ue5-release-engine-test branch]
2021-08-06 12:53:08 -04:00
devin doucette
bfda02d384 DDC: Replaced FRequest with FRequestGroup to handle nested and chained requests more robustly
Any function that may create a request now has a IRequestOwner& parameter, and uses the Begin and End functions on the owner to manage the lifetime of any requests that it creates, as well as using End to invoke the completion callback for any request which has one.

The new FRequestBarrier may be used to block a group from being considered complete in a scope where more requests may be added to it.

#rb Matt.Peters
#rnx
#preflight 6109b5c403d303000144cce5
#preflight 610acf7103d30300016fda94

#ROBOMERGE-SOURCE: CL 17060470 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v850-17047176)

[CL 17060649 by devin doucette in ue5-release-engine-test branch]
2021-08-04 18:08:50 -04:00
devin doucette
c29fdcf532 DDC: Compress payloads using Oodle Mermaid VeryFast
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 16956293 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 16956298 by devin doucette in ue5-release-engine-test branch]
2021-07-26 10:29:10 -04:00
devin doucette
58774329d7 DDC: Exposed the cache key and build status in the build complete callback params
#rb Zousar.Shaker
#rnx
#preflight 60f5c7a78da1560001c68086

#ROBOMERGE-SOURCE: CL 16888763 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16888787 by devin doucette in ue5-release-engine-test branch]
2021-07-19 16:49:25 -04:00
devin doucette
540937d6f0 DDC: Fixed errors with NO_LOGGING
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 16863666 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16863682 by devin doucette in ue5-release-engine-test branch]
2021-07-15 12:30:05 -04:00
devin doucette
f12c9ea467 DDC: Exposed the name of the build job to the build function
#rb Zousar.Shaker
#rnx

#ROBOMERGE-SOURCE: CL 16743715 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16743721 by devin doucette in ue5-release-engine-test branch]
2021-06-22 11:27:03 -04:00
Devin Doucette
97be0c7aff DDC: Added CreateCacheBucket to FBuildConfigContext
#rb Zousar.Shaker
#rnx

[CL 16548191 by Devin Doucette in ue5-main branch]
2021-06-03 11:48:59 -04:00