Commit Graph

4 Commits

Author SHA1 Message Date
Ben Marsh
1ae32843fa Copying //UE4/Dev-Build to //UE4/Dev-Main (Source: //UE4/Dev-Build @ 3058348)
#lockdown Nick.Penwarden
#rb none

==========================
MAJOR FEATURES + CHANGES
==========================

Change 2982033 on 2016/05/18 by Chad.Garyet

	Checking in progress on the ue4 BuildGraph conversion.
	Builds and Compiles editor and game on all platforms
	Builds DDC on win64 and mac
	Builds Tools on win64

Change 3047983 on 2016/07/13 by Ben.Marsh

	PR #2597: Fix P4 environment used for running BuildUGS commandlet (Contributed by paulevans)

Change 3048267 on 2016/07/13 by Ben.Marsh

	BuildGraph: Allow more permissive list of characters for node names; just restrict against characters which are illegal in filenames. Allows creating aggregate names which match job names (eg. "Editor, Tools & Monolithics").

Change 3048293 on 2016/07/13 by Ben.Marsh

	BuildGraph: Allow passing -listonly without a specific -target=... parameter in BuildGraph, to see the contents of the entire script.

Change 3048454 on 2016/07/13 by Ben.Marsh

	BuildGraph: Disable output of error messages when just printing the contents of the graph.

Change 3048507 on 2016/07/13 by Ben.Marsh

	BuildGraph: Rename "Ticket" to "Token" for files used to ensure exclusive access to run part of a build.

Change 3049459 on 2016/07/14 by Matthew.Griffin

	Updated location of HTML5 SDKs for Installed Builds
	#jira UE-32171

Change 3049675 on 2016/07/14 by Matthew.Griffin

	Ensured that all platforms are registered when running -validateplatform command
	#jira UE-31082

Change 3049922 on 2016/07/14 by Ben.Marsh

	UBT: Fix path to XML config file in boilerplate message.

Change 3051483 on 2016/07/15 by Ben.Marsh

	EC: Remove code to prettify node names, now that we can have pretty node names explicitly.

Change 3051522 on 2016/07/15 by Ben.Marsh

	BuildGraph: Change spawn task to fail if a non-zero exit code is returned by an external program. The minimum exit code to be treated as an error can be set using the "ErrorLevel" attribute, similar to ERRORLEVEL in DOS.

Change 3051770 on 2016/07/15 by Ben.Marsh

	UGS: Add support for narrowing virtual streams; fetch event and precompiled binaries for parent stream instead.

Change 3052990 on 2016/07/17 by Ben.Marsh

	Show the names of people with notifications disabled in the heading of failure emails, so it's clear that they're not on CC.

Change 3053556 on 2016/07/18 by Ben.Marsh

	BuildGraph: Add a explicit <Option> tag instead of the <Property Default=""/> shenanigans, so that properties that are meant to be modified by the user are listed explicitly. Supported attributes are "DefaultValue" (which specifies a default if the user does not set it on the command line), "Description" (which explains the purpose of the option to users, which is displayed in a table when BuildGraph is invoked with the -listonly argument), and "Restrict" (which specifies a regex to validate an argument supplied by the user).

	Also add an <EnvVar Name="Blah"/> tag which imports the given environment variable as a property (or sets it to "" if it doesn't exist), and rename the <Choose>/<Option>/<Otherwise> triple to <Switch>/<Case>/<Default> to avoid confusion with the new <Option> tag.

Change 3053688 on 2016/07/18 by Ben.Marsh

	Update build scripts to link to p4-swarm rather than p4-web in dashboard pages and notification emails.

Change 3054039 on 2016/07/18 by Ben.Marsh

	Fix confusing message when compiler isn't installed if the target forces VS2013

Change 3054360 on 2016/07/18 by Ben.Marsh

	Remove GUBP support from EC scripts.

Change 3054399 on 2016/07/18 by Ben.Marsh

	Remove circular include from Json.h -> JsonSerializerMacros.h -> Json.h

Change 3055671 on 2016/07/19 by Ben.Marsh

	Remove incomplete UWP integration from UE4.

Change 3055943 on 2016/07/19 by Ben.Marsh

	Remove the WinRT target platform.

Change 3056270 on 2016/07/19 by Ben.Marsh

	Core: Move VectorRegister.h include to eliminate include dependency on UnrealMathUtility.h

Change 3056390 on 2016/07/19 by Ben.Marsh

	Core: Directly include headers required by default JsonWriter template instantiation.

Change 3057444 on 2016/07/20 by Ben.Marsh

	UBT: Fall back to checking for the VS140COMNTOOLS environment variable if we couldn't determine the Visual Studio installation directory from the registry. Allows using the standalone Visual Studio build tools to compile UE4.

Change 3058337 on 2016/07/20 by Ben.Marsh

	Remove EnvVarsToXML. All target platforms now determine their compile environment directly from the registry.

Change 3058348 on 2016/07/20 by Ben.Marsh

	Disable optimization for all automation projects. They don't generally do anything particularly CPU intensive, and VS2015 optimizations are inhibitive to debugging.

[CL 3058822 by Ben Marsh in Main branch]
2016-07-20 20:25:02 -04:00
Ben Marsh
5fc947ac7e Copying //UE4/Dev-Build to //UE4/Dev-Main (Source: //UE4/Dev-Build @ 3020969)
==========================
MAJOR FEATURES + CHANGES
==========================

Change 2986511 on 2016/05/23 by Ben.Marsh

	UdpMessaging: Force references to symbols in the files containing UDP automated tests. Since plugins are compiled into static libraries before being linked into the executable, the linker will exclude any object files which don't have any referenced symbols. In non-unity builds, or adaptive unity builds with modified test files, this results in the tests not being linked in.

Change 2993274 on 2016/05/27 by Ben.Marsh

	Fix UGS enumerating deleted .target.cs files when trying to detect editor target name.

Change 2994265 on 2016/05/31 by Ben.Marsh

	Add info about setting up CIS integration and zipped editor builds in UGS.

Change 2994275 on 2016/05/31 by Ben.Marsh

	PR #2443: [Unreal Game Sync] Added -project so shortcut or script can hint at the project file to open. (Contributed by paulevans)

Change 2994287 on 2016/05/31 by Ben.Marsh

	UnrealGameSync: Add information about how UGS self-patches and updates.

Change 2996928 on 2016/06/01 by Ben.Marsh

	UnrealGameSync: Fix trying to sync files which are open for branch or move/add. They don't exist on the server yet.

Change 2997619 on 2016/06/02 by Ben.Marsh

	UAT: Fix PRX files not being remapped on PS4. All non-UFS should be allowed to be remapped, and UFS files can be if we're not using a PAK file.

Change 2999769 on 2016/06/03 by Ben.Marsh

	UBT: Fix codepaths which assume that the current user account has a personal folder. The SYSTEM account (which Jenkins defaults to using) does not.

Change 3004879 on 2016/06/07 by Ben.Marsh

	Remove copy of AWSSDK in NotForLicensees folder.

Change 3004902 on 2016/06/07 by Ben.Marsh

	UAT: Switch MCP to use version of AWSSDK that's not in a NotForLicensees folder.

Change 3005892 on 2016/06/08 by Ben.Marsh

	Add the GitHub promotion to the UE4 binary release build.

Change 3016241 on 2016/06/16 by Ben.Marsh

	UGS: Always sync version files at the same changelist as everything else (rather than head revision)

Change 3016446 on 2016/06/16 by Ben.Marsh

	PR #2279: Use MSBuild instead of DevEnv for solution builds (Contributed by FineRedMist)

Change 3016472 on 2016/06/16 by Ben.Marsh

	PR #2442: pointing to the pull requests page of the repo (Contributed by KrishMunot)

Change 3017694 on 2016/06/17 by Ben.Marsh

	EC: Produce an error if trying to sync back to a changelist more than 30 days before the most recent change. Meant to catch errors in entered CL fields from the dashboard. Can be overridden by specifying --allow-old-change in the build arguments.

Change 3017695 on 2016/06/17 by Ben.Marsh

	UBT: Use a well defined order for parsing configuration files, rather than ignoring one file if another has a newer timestamp. Prevents confusing behavior where settings can be present, but are completely ignored. Now prioritizes the BuildConfiguration.xml file in the My Documents/Unreal Engine/UnrealBuildTool, followed by the one in AppData/Roaming/Unreal Engine/UnrealBuildTool. Both are added to the Config section of the solution if present, under different folders.

	#jira UE-24271

Change 3017698 on 2016/06/17 by Ben.Marsh

	Rename the <MsBuild> task to <CsCompile>, highlighting the fact that it only actually works on .csproj files (and not .sln files or other project types).

	#jira UEB-664

Change 3017701 on 2016/06/17 by Ben.Marsh

	BuildGraph: Relax a lot of the restrictions relating to using output tags from nodes. Output tags may contain an arbitrary set of files, including files which are also in other tags or produced by other nodes, but will not be written to temp storage more than once. The default tagged set of files for a node (eg. #MyNodeName) now includes all build products produced by that node.

	Temp storage now separates the storing of build products from the tags which reference them. A TempStorageFileList object is written for each output tag, which includes a list of files as well as a list of the storage blocks referenced by it. When a node depends on a tag, the TempStorageFileList is read first and used to determine which storage blocks to read.

	All tasks now have overloaded functions for returning the tags which they modify and/or reference, and errors are produced if an existing tag is modified, or referenced without being added as an input dependency.

Change 3017714 on 2016/06/17 by Ben.Marsh

	BuildGraph: Allow specifying multiple tag names in the 'Tag' attribute of build tasks (or 'With' attribute of the 'Tag' task).

Change 3018007 on 2016/06/17 by Ben.Marsh

	UBT: Add sections to the target receipt listing files which are required to build using precompiled binaries (as previously generated using -generateexternalfilelist), and runtime dependencies that may be required when using precompiled binaries but aren't actually required for the current target (which previously had to be specified through InstalledEngineFilters.ini). Tested by running UBT with arguments "UE4Game Win64 Development -precompile -xgeexport" and examining target receipt.

	#jira UE-28761

Change 3018322 on 2016/06/17 by Ben.Marsh

	PR #2518: Improvements for Clang on Windows (Contributed by Mattiwatti)

Change 3018365 on 2016/06/17 by Ben.Marsh

	Misc: Fixes for warnings compiling ShaderCompileWorker on Clang

Change 3018397 on 2016/06/17 by Ben.Marsh

	UnrealVS: Add an UnrealVS command to run a single-file-compile with UBT

Change 3019421 on 2016/06/20 by Ben.Marsh

	Fix compilation of FreeType on Clang for Windows - there's an fttypes.h header already in the Windows SDK, so use a relative include path instead.

Change 3019423 on 2016/06/20 by Ben.Marsh

	PR #2518: Improvements for Clang on Windows (Contributed by Mattiwatti)

Change 3020377 on 2016/06/20 by Ben.Marsh

	UBT: Fix strings not being escaped before writing to JSON files.

Change 3020378 on 2016/06/20 by Ben.Marsh

	UBT: Exclude precompiled files from the target receipt which are outside the engine and project directories. We don't need SDK libs being added to the precompile list.

Change 3020966 on 2016/06/21 by Ben.Marsh

	EC: Allow scheduling builds at a certain offset past midnight, using the schedule formatted like "Every 20m from 04:00", and add scheduled builds of target platforms for dev streams every 4 hours. Also make the meaning of "Editor Only" and "Editor, Tools & Monolithics" build names consistent between Dev streams and Main, and add a new "Editor, Tools, Monolithics & DDC" build type includes additional DDC build for //UE4/Main.

#rb none
#lockdown Nick.Penwarden

[CL 3020980 by Ben Marsh in Main branch]
2016-06-21 09:17:49 -04:00
Ben Marsh
c254db71a7 Copying //UE4/Dev-Build to //UE4/Main (Source: //UE4/Dev-Build @ 2944521)
==========================
MAJOR FEATURES + CHANGES
==========================

Change 2909886 on 2016/03/15 by Matthew.Griffin

	Adding a build exception to give a message instead of crashing when trying to generate all project files from an installed build.

Change 2911727 on 2016/03/16 by Matthew.Griffin

	Added Platform Type and Architecture to Installed Platform Info
	Reworked the different IsValid... functions to use lamdas to reduce duplicated code looping and checking receipts
	Moved the code to write config file entries into InstalledPlatformInfo so that it can be reused by anyone wanting to make installed builds
	Added temporary hack to write Android architecture until I can get it from build process

Change 2913692 on 2016/03/17 by Ben.Marsh

	UAT: Move script to archive a build for UGS into a public folder.

Change 2915445 on 2016/03/18 by Ben.Marsh

	UAT: Reduce the number of redundant log warnings/errors after a reported build failure, and simplify calls to ParallelExecutor which don't need retrying.

Change 2915450 on 2016/03/18 by Ben.Marsh

	UAT: Suppress warning messages trying to kill child processes if the operation failed because it's already exited.

Change 2925830 on 2016/03/29 by Matthew.Griffin

	Added new selective download tags
	Added a test for whether installed platforms are missing required files so that we can try to open the launcher to the installer settings

Change 2926437 on 2016/03/29 by Ben.Marsh

	PR #2210: Fix "Rebuild.bat" for paths with parentheses (Contributed by amcofi)

Change 2927399 on 2016/03/30 by Matthew.Griffin

	Updating use of PDBCopy to look in VS2015 folder and fall back to VS2013 version if it doesn't exist.

Change 2933093 on 2016/04/05 by Ben.Marsh

	PR #2232: Updated copyright text to 2016 (Contributed by erikbye)

Change 2936221 on 2016/04/07 by Matthew.Griffin

	Adding checks on architecture for android config options

Change 2938021 on 2016/04/08 by Ben.Marsh

	UAT: Prevent UnauthorizedAccessException when enumerating crash files on Mac from a restricted user account.

Change 2939332 on 2016/04/11 by Matthew.Griffin

	Added AdditionalBundleResources to external file list so that they should be included in Launcher releases

Change 2939767 on 2016/04/11 by Ben.Marsh

	BuildGraph: Add a -preprocess option, which will cause the preprocessed and culled graph out to an XML file for debugging.

Change 2941611 on 2016/04/12 by Ben.Marsh

	UAT: Prevent warning about commands requiring P4 if -p4 is specified on the command line.

Change 2942037 on 2016/04/13 by Ben.Marsh

	UBT: Only print 'Detailed Action Stats' message footer if there were any detailed action stats.

Change 2942640 on 2016/04/13 by Ben.Marsh

	GUBP: Trigger GitHub promotions by triggering a new procedure rather than scanning for labels.

Change 2942728 on 2016/04/13 by Ben.Marsh

	BuildGraph: Rename "AgentGroup" to "Agent" for consistency with XML.

Change 2942735 on 2016/04/13 by Ben.Marsh

	BuildGraph: Few renames to match class names (Build.cs -> BuildGraph.cs, AgentGroup.cs -> Agent.cs)

Change 2943568 on 2016/04/14 by Ben.Marsh

	EC: Print out the log folder at the start of each job.

Change 2944421 on 2016/04/14 by Ben.Marsh

	EC: Add GitHub dashboard page which shows the current syncing state

#lockdown Nick.Penwarden

[CL 2944733 by Ben Marsh in Main branch]
2016-04-14 20:35:31 -04:00
Andrew Grant
505e2440b1 Copying //UE4/Orion-Staging to //UE4/Main (Origin: //Orion/Dev-General @ 2904087)
==========================
MAJOR FEATURES + CHANGES
==========================

#lockdown Nick.Penwarden

Change 2903938 on 2016/03/10 by Frank.Gigliotti

	Added an instance ID to FAnimMontageInstance

	#CodeReview Laurent.Delayen
	#RB Laurent.Delayen
	#Tests PIE

Change 2903745 on 2016/03/10 by Wes.Hunt

	Update Oodle TPS
	#rb none
	#tests none
	#codereview:john.pollard

Change 2903689 on 2016/03/10 by Uriel.Doyon

	New "LogHeroMaterials" console command, displaying the current state of materials and  textures on the character hero.
	#rb marcus.wasmer
	#codereview marcus.wassmer
	#tests editor, playing PC games, trying the new command

Change 2903669 on 2016/03/10 by Aaron.McLeran

	OR-17180 Make stat soundcues and stat soundwaves NOT display zero volume sounds

	- Change only effects debug stat commands for audio guys

	#rb none
	#tests played paragon with new debug stat commands, confirms doesn't show zero-volume sounds

Change 2903625 on 2016/03/10 by John.Pollard

	XB1 Oodle SDK

	#rb none
	#tests none
	#codereview Jeff.Campeau

Change 2903577 on 2016/03/10 by Ben.Marsh

	Remaking latest build scripts from //UE4/Main @ 2900980.

Change 2903560 on 2016/03/10 by Ben.Marsh

	Initial version of BuildGraph scripts - used to create build processes in UE4 which can be run locally or in parallel across a build farm (assuming synchronization and resource allocation implemented by a separate system). Intended to supersede GUBP.

	Build graphs are declared using an XML script using syntax similar to MSBuild, ANT or NAnt, and consist of the following components:

	* Tasks: Building blocks which can be executed as part of the build process. Many predefined tasks are provided (<Cook>, <Compile>, <Copy>, <Stage>, <Log>, <PakFile>, etc...), and additional tasks may be added be declaring classes derived from AutomationTool.CustomTask in other UAT modules.
	* Nodes: A named sequence of tasks which is executed to produce outputs. Nodes may have input dependencies on other nodes before they can be executed. Declared with the <Node> element in scripts.
	* Agent Groups: A set of nodes nodes which is executed on the same machine if running as part of a build system. Has no effect when building locally. Declared with the <Group> element in scripts.
	* Triggers: Container for groups which should only be executed when explicitly triggered (using the -Trigger=<Name> or -SkipTriggers command line argument). Declared with the <Trigger> element in scripts.
	* Notifiers: Specifies email recipients for failures in one or more nodes, whether they should receive notifications on warnings, and so on.

	Properties can be passed in to a script on the command line, or set procedurally with the <Property Name="Foo" Value="Bar"/> syntax. Properties referenced with the $(Property Name) notation are valid within all strings, and will be expanded as macros when the script is read. If a property name is not set explicitly, it defaults to the contents of an environment variable with the same name.

	Local properties, which only affect the scope of the containing XML element (node, group, etc...) are declared with the <Local Name="Foo" Value="Bar"/> element, and will override a similarly named global property for the local property's scope.

	Any elements can be conditionally defined via the "If" attribute, and are largely identical to MSBuild conditions. Literals in conditions may be quoted with single (') or double (") quotes, or an unquoted sequence of letters, digits and underscore characters. All literals are considered identical regardless of how they are declared, and are considered case-insensitive for comparisons (so true equals 'True', equals "TRUE"). Available operators are "==", "!=", "And", "Or", "!", "(...)", "Exists(...)" and "HasTrailingSlash(...)". A full grammar is written up in Condition.cs.

	File manipulation is done using wildcards and tags. Any attribute that accepts a list of files may consist of: a Perforce-style wildcard (matching any number of "...", "*" and "?" patterns in any location), a full path name, or a reference to a tagged collection of files, denoted by prefixing with a '#' character. Files may be added to a tag set using the <Tag> Task, which also allows performing set union/difference style operations. Each node can declare multiple outputs in the form of a list of named tags, which other nodes can then depend on.

	Build graphs may be executed in parallel as part build system. To do so, the initial graph configuration is generated by running with the -Export=<Filename> argument (producing a JSON file listing the nodes and dependencies to execute). Each participating agent should be synced to the same changelist, and UAT should be re-run with the appropriate -Node=<Name> argument. Outputs from different nodes are transferred between agents via shared storage, typically a network share, the path to which can be specified on the command line using the -SharedStorageDir=<Path> argument. Note that the allocation of machines, and coordination between them, is assumed to be managed by an external system.

	A schema for the known set of tasks can be generated by running UAT with the "-Schema=<FileName>" option. Generating a schema and referencing it from a BuildGraph script allows Visual Studio to validate and auto-complete elements as you type.

	#rb none
	#codereview Marc.Audy, Wes.Hunt, Matthew.Griffin, Richard.Fawcett
	#tests local only so far, but not part of any build process yet

Change 2903539 on 2016/03/10 by John.Pollard

	Improve replay playback debugging of character movement

	#rb none
	#tests replays

Change 2903526 on 2016/03/10 by Ben.Marsh

	Remake changes from //UE4/Main without integration history, to add support for BuildGraph tasks.

	#rb none
	#tests none

Change 2903512 on 2016/03/10 by Dan.Youhon

	Modify minimum Duration values for JumpForce and MoveToForce ability tasks so that having minimum Duration values doesn't trigger check()s

	#rb None
	#tests Compiles

Change 2903474 on 2016/03/10 by Marc.Audy

	Fix crash if ChildActor is null
	#rb None
	#tests None

Change 2903314 on 2016/03/10 by Marc.Audy

	Fix ParentComponent not being persisted and fixup content that was saved in the window it was broken
	#rb James.Golding
	#tests Selection of child actors works as expected
	#jira UE-28201

Change 2903298 on 2016/03/10 by Simon.Tovey

	Disabling the trails optimization.

	#tests none
	#rb none

	#codereview Olaf.Piesche

Change 2903124 on 2016/03/10 by Robert.Manuszewski

	Small refactor to pak signing to help with exe protection

	#rb none
	#tests none

[CL 2907678 by Andrew Grant in Main branch]
2016-03-13 18:53:13 -04:00