Commit Graph

1383 Commits

Author SHA1 Message Date
Ben Marsh
efd4603e8c Remove the FakeFail option; never used it, and it's easy enough to add back in if need be.
[CL 2662062 by Ben Marsh in Main branch]
2015-08-19 20:55:29 -04:00
Ben Marsh
c2851c6a68 Move ArchiveBuildProducts() and RetrieveBuildProducts() into the BuildNode base class.
[CL 2661962 by Ben Marsh in Main branch]
2015-08-19 19:27:58 -04:00
Ben Marsh
3cff2cd338 Move build graph into its own class.
[CL 2661953 by Ben Marsh in Main branch]
2015-08-19 19:21:14 -04:00
Ben Marsh
42783dfb6f Separate node templates from their implementations. Prelude to switching towards the build graph being specified through markup.
[CL 2661940 by Ben Marsh in Main branch]
2015-08-19 19:03:19 -04:00
Ben Marsh
582b132551 Add a BuildLocalization node, which runs the UAT Localise script.
#codereview Matt.Kuhlenschmidt, Justin.Sargent

[CL 2661758 by Ben Marsh in Main branch]
2015-08-19 17:27:34 -04:00
Wes Hunt
de3ae73eb3 Reduce temp storage spam. Print UTC time in low when UAT starts.
[CL 2661691 by Wes Hunt in Main branch]
2015-08-19 17:04:29 -04:00
Wes Hunt
9704d5f59f Switching back to Ionic.Zip for temp storage saving because mono is running the macs out of memory.
* The problem is mono stages all files to a memory stream before copying it out, creating intense working memory pressure while creating the zips. By doing this all in parallel, we really hurt the working set size for large storage requests like RootEditor and DDC.
* Also turned back on manifest comparison for remote retrieval, now that timestamp comparison failures won't cause a failure. I'll use the warnings to track down build stages that are redundantly regenerating build artifacts, but we don't need to fail the builds on them.
#codereview:ben.marsh

[CL 2661630 by Wes Hunt in Main branch]
2015-08-19 16:40:57 -04:00
Wes Hunt
adac00f747 Limit parallelism on unzip for mac. Too much memory is being used apparently when it starts up this many threads. :(
[CL 2661336 by Wes Hunt in Main branch]
2015-08-19 14:30:50 -04:00
Ben Marsh
08a11f1fc9 Move RequireFilesToExist out of target receipts; it's just internal state for UAT, and doesn't belong there.
[CL 2661231 by Ben Marsh in Main branch]
2015-08-19 13:35:56 -04:00
Matthew Griffin
a393ec848d Added potential location for third party binaries to the build plugin command
[CL 2661043 by Matthew Griffin in Main branch]
2015-08-19 11:27:49 -04:00
Wes Hunt
e70cb17db2 No, really, for real this time make timestamp mismatches in temp storage tamper check be a warning only.
[CL 2660805 by Wes Hunt in Main branch]
2015-08-19 07:54:15 -04:00
Wes Hunt
ad76e8e6d7 Turn Manifest verification errors into a warning until we can root out what set of redundant work is causing the files to change.
[CL 2660523 by Wes Hunt in Main branch]
2015-08-19 00:26:19 -04:00
Ben Marsh
74eec1cf2a Fix failed pull requests leaving the P4 workspace in a dirty state.
[CL 2660414 by Ben Marsh in Main branch]
2015-08-18 22:03:46 -04:00
Wes Hunt
27654508ca Change temp manifest verification to just simply ignore all timestamps of the set that it was ignoring before. Fixing the bug in manifest comparison to check all directories exposed a ton of additional areas where we were throwing warning on tamper checks, which was causing warnings on PC builds that never used to be there. There will still be warnings in other binaries folders (like plugins) that I am trying to suss out.
* Also removed unnecessary using clauses, which Resharper took as an opportunity to remove redundant qualifiers as well for some reason, thus the larger diff than necessary.
#codereview:ben.marsh

[CL 2660404 by Wes Hunt in Main branch]
2015-08-18 21:51:11 -04:00
Bob Tellez
68db7855bc [AUTOMERGE]
#UE4 Fix to properly pass through extension wildcards when staging debug files, not just the extension.

--------
Integrated using branch Ue4-To-UE4-Fortnite-Simple (reversed) of change#2659601 by Bob.Tellez on 2015/08/18 13:44:18.

[CL 2660027 by Bob Tellez in Main branch]
2015-08-18 17:14:56 -04:00
Bob Tellez
372a8c5232 [AUTOMERGE]
#UE4 dsym files are now staged as DebugNonUFS files so they are excluded from chunking just like pdbs in windows

--------
Integrated using branch Ue4-To-UE4-Fortnite-Simple (reversed) of change#2659582 by Bob.Tellez on 2015/08/18 13:37:14.

[CL 2660025 by Bob Tellez in Main branch]
2015-08-18 17:14:41 -04:00
Daniel Lamb
6bde4efe64 Changed the directory which failed diffed content is saved to.
[CL 2659998 by Daniel Lamb in Main branch]
2015-08-18 16:58:49 -04:00
Daniel Lamb
0b97633583 Enabled diffing cooked builds for orion to help find deterministic cook issues.
[CL 2659995 by Daniel Lamb in Main branch]
2015-08-18 16:56:56 -04:00
Michael Trepka
ec554b04a8 Modified GetParamValueIfNotSpecified to return Default value if the param is not part of the command line
#codereview Robert.Manuszewski

[CL 2659749 by Michael Trepka in Main branch]
2015-08-18 15:17:14 -04:00
Wes Hunt
c094c70740 Fix Zip temp storage functionality on Macs.
* Mono doesn't correctly set the timestamp on files it archives, so we can't rely on the timestamp matching the manifest. Instead we still rely on filesizes, but we only check the timestamps when checking local manifests for tampering.
* Also reduced the logspam I added earlier to help debug this.
* Also fixed a bug in the manifest comparison code that only checked the sizes/timestamps of the first directory in the manfest. This bug has been around forever, so it may expose other issues, but preflight builds looked ok so far.
#codereview:ben.marsh

[CL 2659639 by Wes Hunt in Main branch]
2015-08-18 14:06:15 -04:00
Wes Hunt
093b2c2c60 Increase verbosity level of FixUnixFilePermissions. Too spammy on build logs, and we're not debugging this stuff right now.
[CL 2659633 by Wes Hunt in Main branch]
2015-08-18 14:03:25 -04:00
Ben Marsh
c7355c6ac9 Add a UAT command to sync and build a project from Perforce, similar to UGS.
[CL 2659547 by Ben Marsh in Main branch]
2015-08-18 13:16:16 -04:00
Robert Manuszewski
e70582789a Adding documentation for SkipCookingEditorContent in UAT.
[CL 2659139 by Robert Manuszewski in Main branch]
2015-08-18 07:32:09 -04:00
Ben Marsh
3efd84874f Remove the last GUBPNode dependency from GUBP.cs (the list of build products). Core GUBP now only deals with abstract BuildNode classes.
Also remove methods to handle removing stale DLLs; we don't need to do this now that binaries are not in Perforce, and remove the special-case checks for the VerisonFiles node.

[CL 2657914 by Ben Marsh in Main branch]
2015-08-17 11:30:23 -04:00
Ben Marsh
c2dfa0bb39 Move archiving into a function on BuildNode.
[CL 2657303 by Ben Marsh in Main branch]
2015-08-15 19:40:02 -04:00