Commit Graph

388 Commits

Author SHA1 Message Date
Ben Marsh
e58c6282b6 BuildGraph: Add a better error when an exception is thrown parsing a task argument.
#preflight none

[CL 23464279 by Ben Marsh in ue5-main branch]
2022-12-09 11:22:43 -05:00
David Harvey
68d54e1d19 Build Graph reports which previous steps created any modified build products.
#rb Ryan.Hummer
#rnx
#preflight 6391cad267018b14b57710d8

[CL 23444704 by David Harvey in ue5-main branch]
2022-12-08 06:35:17 -05:00
Alexis Matte
8b8765cb61 Make sure InterchangeWorker is build by UGS and when we release it.
#jira UE-108125
#rb
#rnx
#preflight 638a0c194b2f03a7d7a812fd

[CL 23372363 by Alexis Matte in ue5-main branch]
2022-12-02 09:45:14 -05:00
Wojciech Krywult
74cdb44436 Build: Symbol Server: Added support for source server (source code indexing) for consoles.
Source code indexing allows Visual Studio to automatically fetch the right source code files from Perforce when debugging builds/crashdumps.

Previously, we had this functionality available only on Windows in the form of SrcSrv build task (build graph). This task takes generated pdbs and embeds information about the source code used into them. However, this functionality doesn't translate well into similar features on other platforms.

Resolved by extending SymStore task to handle source indexing in additional to uploading symbols to the symbol store. SrcSrv remains functional, but it still works only on Windows. For this reason, SymStore becomes the preferred solution as it's more general.

Fixed a few additional problems, both in the task itself and our game's build graph.

#preflight 63727a48ee4d25f90ace140b
#rb Robert.Millar
#jira UE-70463

[CL 23123619 by Wojciech Krywult in ue5-main branch]
2022-11-14 14:37:48 -05:00
Ben Marsh
c7c2d972cc Horde: Add another class of blobs for storage tests (20,000 x 1mb).
#preflight none

[CL 22814386 by Ben Marsh in ue5-main branch]
2022-10-27 14:08:38 -04:00
Ben Marsh
150c7bc5d2 Horde: Clean up log output for storage tests.
#preflight none

[CL 22813991 by Ben Marsh in ue5-main branch]
2022-10-27 13:49:55 -04:00
Ben Marsh
0d7929d665 Horde: Add a temp storage test script which generates random data into a given directory.
#preflight none

[CL 22810754 by Ben Marsh in ue5-main branch]
2022-10-27 11:23:16 -04:00
josh adams
30245001cf - Change Mac to use "dotnet msbuild" to compile C# programs with, instead of msbuild - dotnet is shipped with the engine so is always there. This is to fix builders building with xbuild which is deprecated and going away
- Removed unused MobileDeviceInterface
#jira UE-164616
#rb adam.kinge
#preflight 6352ccfee6096564af79599f

[CL 22693841 by josh adams in ue5-main branch]
2022-10-21 13:25:02 -04:00
Ryan Hummer
84a9b6f514 Adding an option to login to any repository by specifying a json with the user & token
#rnx
#jira none
#rb ??
#preflight none

[CL 22525427 by Ryan Hummer in ue5-main branch]
2022-10-14 09:00:31 -04:00
Ben Marsh
d71324777d BuildGraph: Fix temp storage copy actions running 5 times, even if it succeeds on the first attempt.
#fyi Chad.Garyet
#preflight none

[CL 22353978 by Ben Marsh in ue5-main branch]
2022-10-05 09:44:17 -04:00
carl bystrom
f0302df3e2 BuildGraph: Allow overriding path to Docker executable through env var
Allows for using alternative but compatible Docker implementations.

#rb Ben.Marsh
#jira none
#preflight none

[CL 22332338 by carl bystrom in ue5-main branch]
2022-10-04 12:16:09 -04:00
ryan hummer
f6821c1631 Allow the CompileTask to enable AllCores for UBT
Turns on automatically if CommandUtils.IsBuildMachine is true or the CompileTask sets AllowAllCores to true

#rnx
#jira none
#rb none
#preflight 633b3aaf3629a1a4f92f3ef1

[CL 22323425 by ryan hummer in ue5-main branch]
2022-10-03 21:15:14 -04:00
gary yuan
e928117511 Fix BuildGraph ParamHelpAttribute for -target parameter is not really required because -SingleNode could be used instead.
#rb Tim.Kennedy
#preflight 632265c78c478acecfe63f44

[CL 22039972 by gary yuan in ue5-main branch]
2022-09-15 18:09:05 -04:00
gary yuan
8be852fca6 Fix typo in string constant for Attribute metadata on BuildGraph BuildCommand
#rb trivial
#preflight skip

[CL 21555167 by gary yuan in ue5-main branch]
2022-08-24 18:52:30 -04:00
brandon schaefer
0f99d8453f Workaround for tbb.psym colliding from multiple clients copying tbb.psym
#jira UE-161242
#rb none
#preflight none

[CL 21531151 by brandon schaefer in ue5-main branch]
2022-08-23 22:18:21 -04:00
gary yuan
d0af43fb02 #jira: FORT-505939
Extended ParamHelp attribute for BuildCommand to better describe the metadata of parameters for UAT scripts.
Update BuildGraph with ParamHelp attributes.

#rb: Eric.Knapik
#preflight: 6303e331c00af5e294fbb7e9

[CL 21525146 by gary yuan in ue5-main branch]
2022-08-23 19:25:33 -04:00
UnrealBot
73409369c0 Branch snapshot for CL 21319338
[CL 21319338 in ue5-main branch]
2022-08-10 16:03:37 +00:00
Nick Darnell
d75cbc192e Core - Updating the TBB copy location so that the dll and pdb end up in the same place to make generating portable symbols easier.
Build - updating the tempstorage build graph script to ignore additonal pdb build products for tbb.

#jira UE-159742
#preflight 62d5e73ec0180d80715a8b8d

[CL 21161802 by Nick Darnell in ue5-main branch]
2022-07-19 10:00:06 -04:00
Ryan Hummer
dcb81492ef Use robocopy/rsync to copy temp storage manifests locally before unzipping them, and use them to copy manifests off to the share at the end of a task.
improves platform reliability, m1 macs struggle to update file attributes when copying to an smb share. rsync ensures the attributes are correctly copied.

#rnx
#jira none
#rb none
#preflight 62d170c93b057e72748b9aa8
#author ryan.durand

[CL 21110668 by Ryan Hummer in ue5-main branch]
2022-07-15 11:15:30 -04:00
Ben Marsh
6ac2c5b01f UAT: Add a systemic error code for Perforce submit failures.
#preflight none

[CL 21078813 by Ben Marsh in ue5-main branch]
2022-07-13 15:59:41 -04:00
Bryan Johnson
0ba12c9497 Add retry for notary-app calls
#jira UE-155720, UE-157669
#rnx
#rb Ryan.Hummer
#preflight 62c5c25e097ca6afc4f24202

[CL 20968956 by Bryan Johnson in ue5-main branch]
2022-07-06 13:18:50 -04:00
Ben Marsh
38f3bc55ef BuildGraph: Various VM improvements.
* Added support for deserializing BgObject types directly into native classes.
* Removed opcodes for creating graph structures. These are now created in user code from BgObject types.
* Removed BgNodeSpecBuilder. BgNode objects can now be modified directly (returning a modified copy).
* Added concrete types for option parameters. The VM now keeps track of any parameters for evaluated options, allowing them to be added into the graph definition.
* Order dependencies now take nodes rather than outputs.
* Added explicit support for native thunks, whose bindings are saved to a sideband channel during compilation and referenced in bytecode as an index. This generalizes code that was previously specific to node definitions.
* Added a name table to bytecode, to optimize situations where we reference the same string mulitple times.

#preflight 62bf3c583f0d6beee2e8f4a6

[CL 20918762 by Ben Marsh in ue5-main branch]
2022-07-01 14:47:54 -04:00
ryan hummer
f91cb75ed7 Add retrying to potential network share failure points in temp storage.
#rnx
#jira UE-156108, UE-158088
#rb ryan.hummer
#preflight 62bdeccded35ee71a70ee5a5

#ushell-cherrypick of 20574535 by Ryan.Durand

#ROBOMERGE-AUTHOR: ryan.hummer
#ROBOMERGE-SOURCE: CL 20901486 in //UE5/Release-5.0/... via CL 20902831
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v971-20777995)

[CL 20905677 by ryan hummer in ue5-main branch]
2022-06-30 18:10:24 -04:00
Ben Marsh
c32dfb7acb BuildGraph: Move option classes alongside their underlying types, and fix up a lot of namespaces.
#preflight none

[CL 20819013 by Ben Marsh in ue5-main branch]
2022-06-24 19:08:20 -04:00
Ben Marsh
bb0e506aac BuildGraph: Rename classes to reduce boilerplate when writing scripts.
* Expression classes (nodes, aggregates, graphs, etc...) are now called BgNode, BgAggregate, etc...
* Evaluated and instantiated objects used by BuildGraph internals are now called BgNodeDef, BgAggregateDef, etc...

#preflight 62b6374161016695a6545b08

[CL 20818158 by Ben Marsh in ue5-main branch]
2022-06-24 18:30:51 -04:00