Commit Graph

54 Commits

Author SHA1 Message Date
devin doucette
01cb852a23 Deprecated the old names and paths for TSharedString
#rb Zousar.Shaker
#rnx

[CL 34567121 by devin doucette in ue5-main branch]
2024-06-21 12:52:32 -04:00
devin doucette
1a688b31e2 DDC: Fixed Clang static analysis warnings
#jira none
#preflight 640f96f3482188d710688621
#rb Zousar.Shaker
#rnx
#lockdown zousar.shaker

[CL 24643936 by devin doucette in ue5-main branch]
2023-03-14 16:48:35 -04:00
Dan Thompson
8f55b18fe5 Remove debug logging
#fyi devin.doucette
#rb none
#preflight 6408d546cf5fe1e28cea3c19

[CL 24563734 by Dan Thompson in ue5-main branch]
2023-03-08 13:46:10 -05:00
Zousar Shaker
d907347b26 Fix build breakage from CL23917590.
#rb none
#fyi devin.doucette
#preflight skip,BuildBreakFix

[CL 23918382 by Zousar Shaker in ue5-main branch]
2023-01-30 18:34:40 -05:00
Devin Doucette
82621298bf DDC: Added -ExportBuildOutput to include outputs in exported builds
#preflight 63d804445c69f453c111cfd0
#rb Zousar.Shaker

[CL 23917590 by Devin Doucette in ue5-main branch]
2023-01-30 17:44:07 -05:00
Devin Doucette
165d24ede5 DDC: Made the cache optional for the build system
#preflight 636c045ef56cab38c78f8b47
#rb Tim.Smith
#rnx

[CL 23062223 by Devin Doucette in ue5-main branch]
2022-11-09 15:53:21 -05:00
Devin Doucette
381a587dcd DDC: Invoke the build completion callback after storing to the cache
This avoids a race condition where a build completes and then another process is unable to retrieve the data from the cache.

#preflight 636b2a547c2b505190e7d036
#rb Zousar.Shaker
#rnx

[CL 23059312 by Devin Doucette in ue5-main branch]
2022-11-09 13:57:59 -05:00
Devin Doucette
6c33c94b36 DDC: Added experimental interface IBuildSchedulerThreadPoolProvider to integrate the build scheduler with the asset compiling managers
#jira UE-141137
#jira UE-141139
#preflight 63696f5e843e6ac79482dad1
#rb Zousar.Shaker

[CL 23027624 by Devin Doucette in ue5-main branch]
2022-11-08 09:45:50 -05:00
Devin Doucette
92a888c1f6 Replaced uses of _ASV with ANSITEXTVIEW
#preflight 6266acf50634d0904cdfe91c
#rb Steve.Robb
#rnx

[CL 19903251 by Devin Doucette in ue5-main branch]
2022-04-25 11:31:36 -04:00
Devin Doucette
f1c81d6d61 DDC: Reset the build definition later in the build job to ease debugging
#preflight 62582c26044bab1525fca794
#rb Zousar.Shaker
#rnx

[CL 19756364 by Devin Doucette in ue5-main branch]
2022-04-14 10:31:33 -04:00
devin doucette
169318850a DDC: Masked record-only cache policy flags out of the value policy
Fixing backends to respect the masked flags includes a few fixes to handling of ~Query by Put and PutValue.

#rb Matt.Peters
#rnx
#preflight 61f7f7ede55232619f887977

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18790481 in //UE5/Release-5.0/... via CL 18790769 via CL 18791304
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18791515 by devin doucette in ue5-main branch]
2022-01-31 11:52:13 -05:00
devin doucette
517947112b DDC: Reorganized cache policy code into its own files
- FCacheRecordPolicyBuilder now disallows adding value policies with flags that only make sense for records.
- FCacheRecordPolicyBuilder now disallows adding multiple value policies for the same value.
- FCacheRecordPolicyBuilder now skips addition of value policies that match the default policy.

#rb Matt.Peters
#rnx
#preflight 61f471cc6b5aea38e5c64cb6

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18782116 in //UE5/Release-5.0/... via CL 18782123 via CL 18782168
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18782178 by devin doucette in ue5-main branch]
2022-01-28 20:34:31 -05:00
devin doucette
cdedb37d66 DDC: Converted the Build API to UTF-8 where appropriate
#jira UE-133382
#rb Zousar.Shaker
#rnx
#preflight 61e798ee3f00a0a23ef78c49

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18655307 in //UE5/Release-5.0/... via CL 18655320 via CL 18655322
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v900-18638592)

[CL 18655349 by devin doucette in ue5-main branch]
2022-01-19 00:27:48 -05:00
devin doucette
1883382ae7 DDC: Added functions to transform a cache policy and build policy
#rb Zousar.Shaker
#rnx
#preflight 61e0aa6e341d372424e948f1

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18617338 in //UE5/Release-5.0/... via CL 18617349 via CL 18617359
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18617376 by devin doucette in ue5-main branch]
2022-01-14 09:48:40 -05:00
devin doucette
b46e70ea4e DDC: Rename CompleteParams to Response and use FSharedString with no alias
#rb Zousar.Shaker
#rnx
#preflight 61ddf9e0f10add4574e6cf59

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18577467 in //UE5/Release-5.0/... via CL 18577471
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18577481 by devin doucette in ue5-release-engine-test branch]
2022-01-11 17:09:20 -05:00
devin doucette
b4e6fce55b DDC: Changed FCacheRecord to be a collection of values
The structure with a value and attachments is an artifact of an early design where the record *was* a compact binary package. That design changed months ago and it now simplifies the cache record significantly to make it a collection of values without specific "value" and "attachment" types.

#rb Zousar.Shaker
#rnx
#preflight 61d8b94ed17842e547def37a

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18553971 in //UE5/Release-5.0/... via CL 18553976
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18553977 by devin doucette in ue5-release-engine-test branch]
2022-01-07 23:22:29 -05:00
devin doucette
35393bbb2b DDC: Split FPayload into separate FValue and FValueId types
A payload was conceptually a value with an ID. That has been formalized by removing the ID from the payload and having separate FValue and FValueId types. This separation cleans up the API in a few areas, and provides a more natural path to providing a basic key/value API.

#rb Zousar.Shaker
#rnx
#preflight 61d704c04c252480ca284d61

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18531844 in //UE5/Release-5.0/... via CL 18531856
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18531864 by devin doucette in ue5-release-engine-test branch]
2022-01-06 11:05:57 -05:00
devin doucette
48f68493ef DDC: Removed the Put/Get compatibility functions from ICache
#rb Zousar.Shaker
#rnx
#preflight 61d49a971a3fd09dcbc87978

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18511490 in //UE5/Release-5.0/... via CL 18511542
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18511622 by devin doucette in ue5-release-engine-test branch]
2022-01-04 14:38:48 -05:00
devin doucette
27c1393427 CompressedBuffer: Removed partial decompression from FCompressedBuffer now that FCompressedBufferReader is available
Requiring the use of a separate reader type makes it more likely that readers will be reused, and makes it easier to audit reader usage going forward. Reusing readers is desirable to reduce the number of large temporary allocations made during partial decompression of a buffer.

- Added FCompressedBuffer::Save(FArchive&) and renamed FromCompressed(FArchive&) to Load(FArchive&).
- Added FCompressedBufferReaderSourceScope to set a buffer source within a scope.
- Added proper bounds checks to FNoneDecoder.
- Store the header checksum on the decoder context to allow raw blocks to be reused across sources.
- Decode the header on the fly to avoid a temporary header allocation when the header is in contiguous memory.

#rb Zousar.Shaker
#rnx
#preflight 61a98d53800738dbfbc84c73

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18382211 in //UE5/Release-5.0/... via CL 18382310
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v896-18170469)

[CL 18382377 by devin doucette in ue5-release-engine-test branch]
2021-12-06 10:16:05 -05:00
devin doucette
cea98e6ab4 DDC: Split build diagnostics into messages and logs
Messages are deterministic output recorded explicitly by the build function. Logs will be automatically captured from the output log and are not necessarily deterministic. The presence of logs in a build output automatically disables caching of the build output.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18286155 in //UE5/Release-5.0/... via CL 18286173
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18286201 by devin doucette in ue5-release-engine-test branch]
2021-11-24 13:25:25 -05:00
devin doucette
dc4658fa91 DDC: Updated IBuildWorkerExecutor to allow filtering of payloads
#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17876434 in //UE5/Release-5.0/... via CL 17876436
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17876451 by devin doucette in ue5-release-engine-test branch]
2021-10-20 14:50:22 -04:00
devin doucette
01c33e6702 DDC: Reworked FBuildSession to allow filtering of payloads
- Build now takes optional inputs for consistency with BuildAction.
- BuildAction has been renamed to Build and works the same as Build except it starts from an action.
- Build now takes a FBuildPolicy, which is implicitly constructible from EBuildPolicy, and allows setting the policy by payload.
- FBuildContext and FBuildConfigContext now have policy masks that allow clearing flags across every payload in the build.
- EBuildPolicy has CacheStore split into CacheStoreOnQuery and CacheStoreOnBuild for more control of cache behavior.
- Builds now use consistent completion callback parameters, FBuildCompleteParams.
- Build types now have debugger visualization.

#rb Zousar.Shaker
#rnx

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17874670 in //UE5/Release-5.0/... via CL 17874724
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v883-17842818)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17874740 by devin doucette in ue5-release-engine-test branch]
2021-10-20 12:38:01 -04:00
devin doucette
750419503e DDC: Added ECachePolicy::KeepAlive and EBuildPolicy::CacheKeepAlive
The new flags are meant to prevent a cache store from evicting the associated cache records for the duration of the session.

#rb Zousar.Shaker
#rnx
#preflight 614954b611925a00017c525f

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17579887 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17579899 by devin doucette in ue5-release-engine-test branch]
2021-09-21 01:10:55 -04:00
devin doucette
a6226e2f2e DDC: Improved handling of build cancellation
#rb Zousar.Shaker
#rnx
#preflight 6143d4423c7c670001e9aa66

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17551334 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17551348 by devin doucette in ue5-release-engine-test branch]
2021-09-17 09:32:55 -04:00
devin doucette
55ac73b651 DDC: Renamed Dispatch[Op] to Schedule[Op] and tweaked formatting for consistency
#rb none
#rnx
#preflight 613a336abf5a710001674089

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17474273 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)

[CL 17474278 by devin doucette in ue5-release-engine-test branch]
2021-09-09 13:36:48 -04:00