#lockdown Nick.Penwarden
#rb none
==========================
MAJOR FEATURES + CHANGES
==========================
Change 3233612 on 2016/12/13 by Ben.Marsh
UGS: Fix cases where precompiled binaries are submitted for a content change.
Change 3235584 on 2016/12/14 by Ben.Marsh
UBT: Deprecate the overridable TargetRules.ShouldUseSharedBuildEnvironment() function; targets should specify which build environment to use by setting the BuildEnvironment field in their constructor instead.
Change 3235741 on 2016/12/14 by Ben.Marsh
UBT: Deprecate the SetupBinaries() callback on the TargetRules class. Instead of overriding this, targets may override their launch module through the "LaunchModuleName" property in their constructor, and add extra modules to the "ExtraModuleNames" field on the TargetRules object itself.
Change 3238176 on 2016/12/16 by Ben.Marsh
UBT: New XML config file parser. Now reads once at startup and can be applied to instanced objects rather than global fields, and caches parsed output in a binary form for quicker initialization on subsequent runs.
Change 3238249 on 2016/12/16 by Ben.Marsh
UBT: Add attribute-driven command line parser.
Change 3238462 on 2016/12/16 by Ben.Marsh
UBT: Include the Platform, Configuration and Architecture on the TargetRules. Add a constructor which takes the TargetInfo object and deprecate the parameterless one; these fields are currently initialized before invoking the constructor in RulesAssembly.
Change 3238564 on 2016/12/16 by Ben.Marsh
UBT: Deprecate the ModuleRules constructor which takes a TargetInfo argument. Replace it with a read-only wrapper around the target rules instead, so target-specific configuration options can be read without needing to access global static variables. Also require that it's passed to the base class constructor.
Change 3239919 on 2016/12/19 by Ben.Marsh
UBT: Remove the "PreferredSubPlatform" property. This is only used for Windows XP support, which is being retired anyway. Having the target define its own architecture is an awkward contractiction to maintain support for, since the target rules are constructed after the architecture already has been determined.
Change 3240061 on 2016/12/19 by Ben.Marsh
UBT: Remove ThirdPartyHeaderFinder. I don't think anything is using this any more.
Change 3240175 on 2016/12/19 by Ben.Marsh
UBT: Add the target name and project file location to the target rules.
Change 3240490 on 2016/12/19 by Ben.Marsh
UAT: Remove Mac staging code that requires UAT to compile and construct .target.cs files when packaging, to check bUseSteam and bUseCEF3 flags. libsteam_api.dylib is now staged at build time by adding it as a bundle resource, CEF3 is already marked as a runtime dependency by CEF3.build.cs, and UnrealCEFSubProcess.app is already marked as a runtime dependency from WebBrowser.build.cs.
Change 3240717 on 2016/12/20 by Ben.Marsh
UAT: Allow configuring UAT to run as if on a build machine by passing the -buildmachine argument on the command line.
Change 3240718 on 2016/12/20 by Ben.Marsh
UAT: Remove target-specific properties that are no longer used by staging (bUsesSlate, bUsesCEF3, etc...). This stuff should all be handled inside UBT.
Change 3241002 on 2016/12/20 by Ben.Marsh
UBT: Move most target configuration settings from UEBuildConfiguration to TargetRules. This encapsulates target specific settings that will allow instancing multiple targets in the future. To facilitate migration of settings to their new location, "UEBuildConfiguration" is now a property that returns the current target rules instance.
Change 3241027 on 2016/12/20 by Ben.Marsh
Only copy the Steamworks dylib into the app bundle for monolithic builds. For all other times, just add it as a runtime dependency.
Change 3241055 on 2016/12/20 by Ben.Marsh
UBT: Only store a ReadOnlyTargetRules object on the instanced build target; finalize the configuration before it's instantiated.
Change 3241156 on 2016/12/20 by Ben.Marsh
Update SwarmInterface.csproj to .NET framework 4.5, to match what all other C# tools are using.
Change 3241205 on 2016/12/20 by Ben.Marsh
Replace all uses of TargetRules.TargetType with TargetType.
Change 3241881 on 2016/12/21 by Ben.Marsh
UBT: Remove project file information from UEBuildContext base class. As configurable properties are moved on to the TargetRules object, it will no longer be a dependency to instantiate the platform with this path.
Change 3241895 on 2016/12/21 by Ben.Marsh
UBT: Remove toolchain support for Windows XP.
Change 3241908 on 2016/12/21 by Ben.Marsh
UBT: Move settings for windows targets into a WindowsTargetRules class, which is exposed for targets to modify via the TargetRules.WindowsPlatform field.
Change 3242835 on 2016/12/22 by Ben.Marsh
UBT: Fix multiple binaries being created if plugins specify module names more than once. MfMedia.uplugin has MfMediaFactory listed twice.
Change 3242837 on 2016/12/22 by Ben.Marsh
Change UE4EditorServices to include MacSystemIncludes rather than Cocoa.h directly; causes FVector to be defined incorrectly in non-unity builds.
Change 3242923 on 2016/12/22 by Ben.Marsh
Build: Fixes for conforming incremental workspaces:
* P4 have table is now reset before deleting all the files. This prevents Perforce being out of sync if the delete fails for some reason.
* Incremental workspaces are skipped when running a clean operation, because they do not have a workspace capture file (causing a full re-sync, always). The "P4 Clean" option is still effective for cleaning these workspaces.
Change 3242961 on 2016/12/22 by Ben.Marsh
UBT: Move entry points from PreBuildSync() and PostBuildSync() from UEToolChain to UEBuildPlatform, and make the original toolchain versions static. These functions are already at odds with the rest of the data flow within UBT because they rely on global state cached outside the toolchain instance, making assumptions that UBT is only ever going to be invoked with one target that's constructed in the same run and that nothing is being cached (eg. UBT makefiles). Moving them onto UEBuildPlatform simplifies the toolchain lifecycle for other platforms without leaving Mac and IOS any more limited than they were before.
Change 3242981 on 2016/12/22 by Ben.Marsh
UBT: Remove symbol server handling code into UAT's platform library, since it's never needed from inside UBT.
Change 3242999 on 2016/12/22 by Ben.Marsh
UBT: Remove the StripSymbols() function from the UEToolChain base class. This functionality is only required by UAT, so it's better suited to being exposed through UAT's modular platform classes.
Change 3243022 on 2016/12/22 by Ben.Marsh
UBT: Require an instance of the target rules to be able to construct a toolchain in UBT. This will allow configuring toolchain-specific options from the target, using reflection from config files, and the command line.
Change 3243083 on 2016/12/22 by Ben.Marsh
UBT: Move settings for the Windows compiler version to use onto the Windows-specific target rules.
Change 3243090 on 2016/12/22 by Ben.Marsh
UBT: Change the third party paths in UEBuildConfiguration to constants. Changing these would not work.
Change 3243423 on 2016/12/23 by Ben.Marsh
UBT: Move a lot of settings from BuildConfiguration to TargetRules. This allows different targets to have different settings, naturally, and moves converts argument parsing and config to be driven by attributes.
Change 3243516 on 2016/12/23 by Ben.Marsh
UBT: Remove the ValidateUEBuildConfiguration callback, which is no longer used. (XGE settings validation occurs in XGE.cs)
Change 3244020 on 2016/12/28 by Ben.Marsh
UBT: Remove the BaseIntermediatePath static property. Precursor to removing RelativeEnginePath and IntermediateFolder properties.
Change 3244074 on 2016/12/28 by Ben.Marsh
UBT: Remove the RelativeEnginePath variable from BuildConfiguration. UnrealBuildTool.EngineDirectory gives the absolute path, and can be used to construct a relative path when necessary.
Change 3244076 on 2016/12/28 by Ben.Marsh
UBT: Remove BuildConfiguration.BaseIntermediateFolder; just use a fixed directory everywhere instead.
Change 3244083 on 2016/12/28 by Ben.Marsh
UBT: Replace FileReference and DirectoryReference instance methods for manipulating files and directories with static methods, to mirror the System.IO.File and System.IO.Directory classes.
Change 3244441 on 2016/12/31 by Ben.Marsh
UBT: Remove code to force PDBs when building with no debug info under XGE. Verified described symptoms (that it causes PCH generation to be serialized) no longer occur.
Change 3244687 on 2017/01/03 by Matthew.Griffin
Changed Exception to use FirstInclude.IncludeName as PrecompiledHeaderIncludeFilename can be null when this occurs
Change 3246112 on 2017/01/04 by Ben.Marsh
UBT: Fix UHT failures building some targets with the -useprecompiled option, due to differences in the order that circularly dependent modules are parsed. Precompiled binaries are now kept in the regular AppBinaries list, but are excluded from the build at the last minute. Also change some checks from IsEngineInstalled() to bUsePrecompiled, to prevent headers being overwritten when running in a non-installed precompiled build.
Change 3246223 on 2017/01/04 by Ben.Marsh
UBT: Prevent version manifests being overridden if a file is not being built as part of the target.
Change 3246387 on 2017/01/04 by Ben.Marsh
UBT: Remove BuildConfiguration settings for UnrealCodeAnalyzer. This tool isn't used at the moment, but it's configured using global variables accessed from all over the UBT codebase, making it difficult to refactor the build options into an instanced object. If we bring this tool back from the dead in the future, it should be possible to implement it using the exported JSON target definition or the XGE manifest, similarly to how IncludeTool uses it.
Change 3247004 on 2017/01/04 by Ben.Marsh
UBT: Simplify the logic for cleaning targets in UBT. Now uses FileReference/DirectoryReference objects everywhere, doesn't require the compile/link environment, and does all the checking to avoid deleting precompiled binaries in one place.
Change 3247250 on 2017/01/04 by Ben.Marsh
UBT: Prevent precompiled binaries being added to the list of app binaries twice.
Change 3247594 on 2017/01/05 by Ben.Marsh
Build: Run sample and template editors on the same agents as the other incremental builds. Remove ProtoStar, which does not have any non-precompiled editor target to build.
Change 3247763 on 2017/01/05 by Ben.Marsh
UBT: Allow the toolchain to update the list of build products for each module linked into a binary. Allows Mac to add dylibs and bundle resources specified per-module without having to construct a link environment and try to link them.
Change 3247775 on 2017/01/05 by Ben.Marsh
UBT: Instance the target compile and link environments when they are required during building, and don't persist them on the UEBuildTarget instance.
Change 3247811 on 2017/01/05 by Ben.Marsh
EC: Add a batch file for testing postp filters.
Change 3247839 on 2017/01/05 by Ben.Marsh
EC: Include the name of the file being compiled when parsing MSVC errors and warnings.
Change 3248101 on 2017/01/05 by Ben.Marsh
UBT: Fix Android support for force included headers.
Change 3248533 on 2017/01/05 by Ben.Marsh
PR #3097: UBT project supports optional platforms (Contributed by PrimalJohnScott)
Change 3249205 on 2017/01/06 by Ben.Marsh
UAT: Fix ParseTaggedP4Output throwing an exception if the same key name is specified more than once. This can happen when parsing the output from "P4 INFO", where multiple brokers are present.
Change 3249249 on 2017/01/06 by Ben.Marsh
UBT: Check for the existance of AndroidManifest.xml within extracted AAR directories, rather than just checking for the existance of the directory itself. Perforce does not remove empty directories when cleaning a workspace unless the rmdir option is on the workspace, so this can cause incremental build failures to fail on build machines.
Change 3249486 on 2017/01/06 by Ben.Marsh
UBT: Use relative paths in unity files when compiling for Mac/IOS, rather than generating a separate local/remote version of the file for gathering include dependencies. Absolute paths are only used to work around the way that MSVC concatenates paths internally; we don't hit the same problems when checking dependencies.
Change 3249736 on 2017/01/06 by Ben.Marsh
UBT: Rename CPPEnvironment to CppCompileEnvironment, and remove the separate CPPEnvironmentConfiguration object. All settings are now stored directly on the CppCompileEnvironment object.
Change 3250179 on 2017/01/07 by Ben.Marsh
Fix creating installed build when root directory contains a space in the name.
Change 3250181 on 2017/01/07 by Ben.Marsh
UBT: Remove some esoteric (and unused, AFAIK) options for orthogonally building different platforms.
Change 3250223 on 2017/01/07 by Ben.Marsh
UBT: Merge the LinkEnvironment and LinkEnvironmentConfiguration classes together.
Change 3250233 on 2017/01/07 by Ben.Marsh
UGS: Allow specifying a workspace-specific sync filter, which is applied on top of the standard filter. Also fix filter being cleared if the cancel button is pressed, and help text being stripped out.
Change 3250241 on 2017/01/07 by Ben.Marsh
UBT: Move the options for specifying additional Android architectures to target onto an Android-specific object on the TargetRules.
Change 3250400 on 2017/01/08 by Ben.Marsh
UBT: Move executor config settings onto the executor instances.
Change 3257708 on 2017/01/13 by Ben.Marsh
UBT: Remove the ThirdPartySourceDirectory constant; there are many places which hard-code or assume this location anyway, and it's not going to change.
Change 3260535 on 2017/01/17 by Ben.Marsh
Add an optional "RequiredSubmittedChange" setting to EC settings files. Allows a scheduled job to run even if there have been no code changes submitted. Test with the utilization capture job.
Change 3260875 on 2017/01/17 by Ben.Marsh
EC: Fix workspaces getting out of sync wrt. newly added files when jobs are aborted during a sync. In such cases, the P4 have table indicates the new files have been synced locally, but the workspace is forced back to a state before it had them due to the capture file. When a following sync tries to add them again, P4 believes the workspace already has them synced.
To work around this, we now write an additional file to the root folder of a workspace containing the last CL that was captured, and sync back to it before doing the reconcile.
Change 3261724 on 2017/01/18 by Ben.Marsh
Allow filtering job types from the list view in EC. Hide the utilization capture job by default. Also set up notifications for the utilization capture job.
Change 3261756 on 2017/01/18 by Ben.Marsh
IncludeTool: Prevent matching a full enum declaration as a forward declaration.
Change 3261932 on 2017/01/18 by Ben.Marsh
EC: Add support for specifying days of the week in schedules. The following syntaxes are supported:
"Monday, Tuesday and Wednesday at 10:30"
"Daily except Sunday and Wednesday at 14:30"
Also tweak display of dates relative to now to handle dates/times in the future, and include the date when specifying a day name.
#jira UEB-729
Change 3262676 on 2017/01/18 by Ben.Marsh
UBT: Split UBTMakefile into its own file. (From PR #3106)
Change 3263893 on 2017/01/19 by Ben.Marsh
UBT: Stop exporting platform classes from UBT, as well as all the referenced classes that have to be made public as a result. Any platform-specific functionality that needs to be shared with UAT is now exposed through wrappers in separate public classes, eg. WindowsExports.cs, IOSExports.cs, etc...
Change 3264291 on 2017/01/19 by Ben.Marsh
UBT: Fix errors generating documentation in UBT, and enable it by default. Will catch more errors with new code being added. Originally in PR #3106, but redone due to conflicts.
Change 3264534 on 2017/01/19 by Ben.Marsh
UBT: Include plugin config files in generated projects.
Change 3264571 on 2017/01/19 by Ben.Marsh
UBT: Prevent overwriting .modules files if nothing has changed. On builders, it's common to build multiple editors in the same workspace, and changing the last modified timestamp causes BuildGraph to fail due to tampered files.
Change 3265745 on 2017/01/20 by Ben.Marsh
UGS: Automatically open UGS when running the launcher for a second time, rather than prompting to close the original instance.
Change 3265777 on 2017/01/20 by Ben.Marsh
UGS: Automatically close and reopen when UGS is re-ran with the shift key held down to switch into unstable mode.
Change 3268314 on 2017/01/23 by Ben.Marsh
UBT: Make sure version manifests are stable by sorting the list of build products, so they are only touched if the contents have really changed.
Change 3269601 on 2017/01/24 by Ben.Marsh
UBT: Fix symbol files being added to manifest for some platforms even though debug info is disabled.
Change 3269607 on 2017/01/24 by Ben.Marsh
UBT: Fix bug where UBT would need to be invoked when switching between two editors sharing the same engine binaries on Mac. The location of the .modules file cannot be guessed on Mac by looking in the same directory as the primary output executable because it's an .app bundle, and the actual modules are nested several directories below that.
Change 3269608 on 2017/01/24 by Ben.Marsh
UBT: Fix additional files copied into the app bundle always being updated on Mac. Now uses rsync --checksum to make sure only modified files are updated.
Change 3271062 on 2017/01/24 by Ben.Marsh
UBT: Fixes for bugs detected by PVS Studio (PR #3161)
Change 3272421 on 2017/01/25 by Ben.Marsh
Fix commends regarding DDC in BaseEngine.ini
#jira UE-41076
Change 3272810 on 2017/01/25 by Ben.Marsh
Fix VS2017 being displayed as 'Visual Studio 15' in the Windows target settings panel.
Change 3272935 on 2017/01/25 by Ben.Marsh
Fix Metal errors launching on Mac due to use of OSX environment settings before they are initialized. Toolchain settings are now constructed on demand in a separate class, for Mac, iOS and TVOS.
Change 3274167 on 2017/01/26 by Ben.Marsh
Fix resource files not being compiled in installed builds on Windows. Was causing metadata not to be embedded into executables.
#jira UE-36457
Change 3275557 on 2017/01/27 by Ben.Marsh
Expand checks for propagation of restricted folder names to include source files, and to ensure that each restricted folder is represented in the output. Also improve messaging to show the dependency chain leading to a restricted folder being referenced, and which folder it is.
Change 3275628 on 2017/01/27 by Ben.Marsh
UBT: Splitting configuration files into one class per-file.
Change 3276784 on 2017/01/29 by Ben.Marsh
Add an authoritative list of confidential folder names, and expose it through global BuildGraph properties ($(RestrictedFolderNames) and $(RestrictedFolderFilter)). Also switch existing scripts to use it.
Change 3276792 on 2017/01/29 by Ben.Marsh
UBT: Use UE4CSharp.prog files to indicate which projects should be included in the solution without having to hard-code a list of them in UBT.
Change 3277263 on 2017/01/30 by Ben.Marsh
IncludeTool: Merging various fixes.
* Fix warnings about #include directives after first code block from parsing monolithic headers.
* Fix exception on startup if the intermediate directory does not already exist.
* Add a special case for ignoring missing header guards from MonolithicHeaderBoilerplate.h, rather than marking it as an inline header. Marking it as inline prevents parsing include directives, which results in including CoreTypes.h from the wrong location.
* Create job objects for spawned compiler instances to prevent them trying allocating more memory than the system can spare.
* Remove (unused) code which makes assumptions about files ending with "Classes.h".
* Add a verbose per-file output log to aid with debugging.
* Negate the MakeStandalone command line option, which was added to allow tweaking forward declarations in already optimized files, so the optimized output does not have missing headers by default.
* Fix missing headers when creating standalone files, due to incorrect list of previous files being passed in to the OutputFile constructor. Now passes the original list of included files, not the output list.
* Fix initial header for a cpp file sometimes being removed. Forcibly including a header at the start of the file does not use the normal pathway for spidering through includes, so a second include of the same file was being generated. Any includes of that header were being forced into output, and the earlier include was then removed due to being redundant.
* Prevent forward declaring enums which have to be parsed by UHT. UHT relies on includes to determine a parse order, and will fail if the enum definition has not been parsed first.
* Use a relative path for private includes in the same module if there are any. Fixes some incorrect paths, and makes it clearer that we're doing something we shouldn't.
Change 3277307 on 2017/01/30 by Ben.Marsh
UBT: Fix private PCHs not using correct header. Causes custom definitions to be excluded from the command line.
[CL 3278101 by Ben Marsh in Main branch]
==========================
MAJOR FEATURES + CHANGES
==========================
Change 2963214 on 2016/05/02 by Ben.Marsh
BuildGraph: Allow specifying optional dependencies for a node, indicating that the build products from an upstream node are desired, but should not block the node from running.
Change 2972295 on 2016/05/10 by Ben.Marsh
EC: Remove spacing in notification emails to reduce size, and help prevent gmail from truncating messages. Also allow mailing notification emails when doing a dry run, and reading stream settings from another branch.
Change 2976096 on 2016/05/12 by Ben.Marsh
EC: Store properties for the last succeeded builds, including the list of users that were notified about it.
Change 2976390 on 2016/05/12 by Ben.Marsh
EC: Add a separate line to the notification email summary with a link to edit settings, and pass the missing ec-update parameter to set the last build status.
Change 2976441 on 2016/05/12 by Ben.Marsh
UAT: Remove log file copy on builders after UAT failure. This is done outside the EC step that originally did it now.
Change 2976456 on 2016/05/12 by Ben.Marsh
BuildGraph: Catch exceptions thrown by child processes failing when building or running UAT commands, and return failure normally without dumping callstacks.
Change 2978440 on 2016/05/16 by Ben.Marsh
EC: Age out entries from the "latest builds" list after a week. There's no obvious way to tell if a node has been removed, but a periodic cleanup should keep the build notifications list in check.
Change 2979446 on 2016/05/16 by Ben.Marsh
Rename ambiguous headers which exist with the same name in different paths.
Change 2979839 on 2016/05/16 by Ben.Marsh
UE4: Renaming HTML5 SocketSubsystem files to eliminate ambiguities.
Change 2979852 on 2016/05/16 by Ben.Marsh
UE4: Use explicit relative paths for public headers in PortalServiceInterfaces modules which do not have unique names
Change 2980113 on 2016/05/17 by Ben.Marsh
UE4: Fix include paths for HTML5 SocketSubsystem files.
Change 2980117 on 2016/05/17 by Ben.Marsh
UE4: Remove reference to private PCH from Oculus common code.
Change 2980186 on 2016/05/17 by Ben.Marsh
UAT: Add a -StopOnErrors parameter to UE4Build, which is propagated to XGE.
Change 2980879 on 2016/05/17 by Ben.Marsh
UE4: Fixup Lightmass to use LightmassPCH.h rather than stdafx.h
Change 2981117 on 2016/05/17 by Ben.Marsh
Portal: Use a unique name for the Portal PCH, rather than just calling it PrivatePCH.h
Change 2981839 on 2016/05/18 by Ben.Marsh
Replace ambiguous D3D11/D3D12 includes with direct includes for the current platform.
#lockdown Nick.Penwarden
[CL 2982178 by Ben Marsh in Main branch]
==========================
MAJOR FEATURES + CHANGES
==========================
Change 2959679 on 2016/04/28 by Ben.Marsh
UGS: Show the original author for changes with a #ROBOMERGE-AUTHOR tag.
Change 2959695 on 2016/04/28 by Ben.Marsh
UGS: Only filter out changes from by buildmachine that contain the string "CIS Counter".
Change 2960798 on 2016/04/29 by Ben.Marsh
Remove C++ version of ParallelExecutor. Now implemented in C# as part of UAT.
Change 2960928 on 2016/04/29 by Ben.Marsh
UGS: Change filter for buildmachine changes to only include rebuilt lightmaps.
Change 2963214 on 2016/05/02 by Ben.Marsh
BuildGraph: Allow specifying optional dependencies for a node, indicating that the build products from an upstream node are desired, but should not block the node from running.
Change 2964454 on 2016/05/03 by Ben.Marsh
Change PostBuildInfoTool to PostBadgeStatus, and add position-independent argument parsing.
Change 2964533 on 2016/05/03 by Ben.Marsh
BuildGraph: Add the ability to generate summary badges from BuildGraph scripts, which can be pushed into a separate database for consumption by UGS.
Change 2964852 on 2016/05/03 by Ben.Marsh
BuildGraph: Add a task which can submit a set of files to Perforce, optionally creating and using a different workspace to do so.
Change 2966856 on 2016/05/04 by Ben.Marsh
EC: Allow specifying a filter for the changes considered when looking for the most recent change. Allows filtering out content changes for UGS builds, code-only builds, etc...
Change 2966867 on 2016/05/04 by Ben.Marsh
EC: Restore code to always set time in CIS state; we never want large builds to trigger off their defined interval.
Change 2967504 on 2016/05/05 by Ben.Marsh
UAT: Make sure the intermediate directory exists before writing out the list of changes in StreamCopyDescription.
Change 2967778 on 2016/05/05 by Ben.Marsh
UAT: Detect the P4 environment by querying Perforce for the setting of P4PORT, rather than assuming it's set in an environment variable. Windows stores this setting in the registry rather than the environment, but it's also valid to be set via P4CONFIG.
Change 2967815 on 2016/05/05 by Ben.Marsh
EC: Copy the initial resource pool setting from the stream settings into an EC property
Change 2967873 on 2016/05/05 by Ben.Marsh
EC: Allow stream settings to be stored directly in /GUBP_V5/Streams/ rather than having to be in a child property sheet.
Change 2969294 on 2016/05/06 by Ben.Marsh
EC: Extend ConformResources command to allow updating the pools that resources are assigned to, and to limit the number of machines which are syncing at once. Also added new EC procedure to allow specifying these arguments.
Change 2969371 on 2016/05/06 by Ben.Marsh
EC: Allow overriding the stream and workspace identifier synced by the builders. Overriding the stream allows syncing a narrower view of files (eg. Dev-Main vs Main), and overriding the workspace identifier allows sharing a workspace between two streams.
Change 2970623 on 2016/05/09 by Ben.Marsh
UAT: Prevent Ctrl-C handler delegate from being garbage collected and failing to be triggered.
Change 2970627 on 2016/05/09 by Ben.Marsh
UAT: Don't limit the list of valid target platforms specified on the command line to just those that we have initialized. Ignoring the platform if the SDK is not installed is never what the user wants.
Change 2972140 on 2016/05/10 by Ben.Marsh
Change 'Engine, Tools and Monolithics' to include QAGame and Template editors, but exclude everything downstream of a trigger.
#lockdown Nick.Penwarden
[CL 2972146 by Ben Marsh in Main branch]
==========================
MAJOR FEATURES + CHANGES
==========================
Change 2790858 on 2015/12/04 by Ben.Marsh
Fix ERRORLEVEL not being correctly returned by build.bat, causing failed builds to be treated as succeeded builds when launched from Visual Studio.
#codereview Wes.Hunt
Change 2792683 on 2015/12/07 by Matthew.Griffin
Removed some usages of RunningRocket() function
Checks for whether engine tools need rebuilding should be ignored in all installed engine builds.
New Intermediate folders should be within project directory and engine's should not be removed when engine installed.
Change 2794194 on 2015/12/08 by Matthew.Griffin
Removed IsRocket function.
Changed IsEngineInstalled so that -NotInstalledEngine will always disable installed engine behavior.
Change 2801483 on 2015/12/14 by Matthew.Griffin
Removed uses of UnrealBuildTool.RunningRocket()
Mac Tool chain test was down to Engine libraries being pre-compiled
Allow Third Party dependencies in any monolithic target, not just games
All installed engine builds should use shared build environment
Removed test no longer needed now that projects from installed builds have game targets
Include engine source if it exists in any installed build to improve intellisense in VCProjects
Change 2803276 on 2015/12/15 by Matthew.Griffin
Removed Rocket switch from ProjectParams as it can always check Automation.RunningRocket()
Removed bGeneratingRocketProjectFiles from ProjectFileGenerator as it can always check UnrealBuildTool.RunningRocket()
Changed a few usages to IsEngineInstalled where obvious
Change 2814190 on 2016/01/03 by Ben.Marsh
Convert ParallelExecutor to managed code, so we can use it trivially from UAT without having to compile with UBT first. Should be moved into UBT at some point.
Also add a support class for creating managed child processes, which are terminated automatically if the parent process is closed. Currently only implemented on Windows, using P/Invoke. Also allows setting priority level of the process, which could replace the hacky way that we determine the physical processor count in UBT.
Change 2822503 on 2016/01/10 by Wes.Hunt
AnalyticsET now uses the Data Router protocol.
* Automatically detects configs that try to use the old endpoint and routes them to the new endpoint. This code is temp until internal games merge the code and have a chance to update their configs.
* There is a new attribute on BeginSession called "LegacyURL" that is "true" if the code needs to have its configs updated, as well as a LogAnalytics warning.
* Updated EngineAnalytics and AutomationAnalytics providers to use the new URL and specify any appropriate new, optional config values.
* Update CrashReporter provider to use new URL and simplified it's provider configuration (now uses AnalyticsET config struct directly).
Removed recently added GetUE4TypeOverride and moved it to the one place in Engine that needs it.
#jira UE-21755
#lockdown nick.penwarden
[CL 2822983 by Wes Hunt in Main branch]
* Moving Ionic.Zip source code into UE4 from UE4 source.
* Only one left should be in Binaries/DotNET.
* Moving TPS info to source location.
* Deleting several copies that were floating around.
Assembly Resolve Refactor
* Added AssemblyUtils.InstallAssemblyResolver to handle resolving of known assemblies that may not exist in the same folder as the referencing assembly.
* This is now installed by UAT and UBT, which should handle all needs to load Ionic.Zip and RPCUtility.exe from scripts that install into subfolders of Binaries/DotNET.
* Other assemblies can be added easily as necesary, centralizing the location where this is handled.
* Removed AssemblyResolver from RPCUtilHelper as UBT handles it automatically now.
* Removed Ionic.Zip references from projects that weren't really using it.
#codereview:ben.marsh
[CL 2646891 by Wes Hunt in Main branch]
GUBP High Level
* Temp Storage is zipped into a single archive per node now. This results in ~75% reduction in temp storage usage and network traffic, not to mention the per-file overhead.
* Temp Storage is in P:\\Builds\\{Game}\\TmpStore instead of P:\\Builds\\{Game}\\GUBP (to facilitate easier cleaning of this new structure).
* Temp Storage nodes are in subdirectories of {Branch}\\{CL}\\{NodeName} now instead of a flat directory structure that was hard to manually sift through.
GUBP Mid Level
* Removed -Store= and -StoreSuffix= test parameters.
* Added -NoZipTempStorage parameter to turn off temp storage zipping if necessary.
* Created GUBP.JobInfo class that collects info about the job as a whole to be passed around by GUBP. Mostly used by any code that need to interact with TempStorage.
* Created TempStorageNodeInfo that describes the necessary parameters to find the temp storage location for a node.
* Fully XML commented TempStorage.cs, and commented internals all major functions.
* Added a bunch of telemetry data for storing, retrieving, and cleaning shared temp storage.
UAT Mid Level
* Fixed a bug in Ionic.Zip that make ExtractAll() not work on Mono, checked in new DLLs.
* Added UAT parameter -UseLocalBuildStorage that allows you to test build storage stuff completely locally. Writes to Engine\\Saved\\LocalBuilds\\...
GUBP Low Level
* Refactored some GUBP startup code so temp vars would be limited in scope. Makes it easier to track the impact of refactoring these things.
* CullNodesForPreflight is only called for preflight builds.
* Refactored TempStorage.FindTempStorageManifests to use new TmpStore structure and harden the brittle string/path parsing it was doing. See the new TempStorage FindMatchingSharedTempStorageNodeCLs().
* Refactored TempStorage Saving and Loading to use XDocument instead of older XmlDocument. Removed a bunch of redundant checks.
* Use StripBaseDirectory and MakeRerootedFilePath to remove the brittle directory manipulation code. Directories no longer require a '/' at the end.
* Removed a few redundant caching layers in cleaning temp storage that try to ensure we don't clean a folder twice. None of them were necessary.
* Removed unused single-threaded copy code from temp storage.
* Updated Temp Storage unit test, and fully commented the logic behind it.
UAT Low Level
* UAT top level exception handler is now a single log line now to help parsers find the error.
* Removed several uses of FormatException as it doesn't display the entire exception chain, and is not as good as the default exception formatter.
* Removed ExceptionToString as it used FormatException, which was not a good precedent.
* Fixed several cases of exception propagation that was not properly chaining the inner exception.
* Refactored ThreadedCopyFiles to use Parallel.For because it was just as fast (if not faster) and much simpler to maintain.
* Removed the suffix from Robust_FileExists_NoExceptions because it's sole purpose in life WAS to throw exceptions!
* Added a bunch of XML doc comments to CommandUtils.
* Modernized some container manipulation and iteration to use IEnumerable and extension methods more appropriately.
* Added several @todos for other minor cleanup stuff that should happen eventually.
* Fixed some uses of String.Compare to use invariant culture.
#codereview:ben.marsh
[CL 2644846 by Wes Hunt in Main branch]
* As of .NET 4.5, the default for Any CPU is to "prefer 32-bit process", and you need to explicitly turn this off to require the process be run as 64-bit.
* See http://blogs.microsoft.co.il/sasha/2012/04/04/what-anycpu-really-means-as-of-net-45-and-visual-studio-11/ for details.
* UAT will now assert when Environment.Is64BitProcess is false on startup, as we essentially require 64-bit for certain scripts, and that's how we ran it before upgrading to .NET4.5.
* Also set default optimization options for the project, which were changed at some point.
* Finally, rolled back CL#2633880, which was trying to work around the fact that UAT was running 32-bit.
#codereview:richard.fawcett, ben.marsh
[CL 2634273 by Wes Hunt in Main branch]
we now use an enum similar to UE4 with Fatal, Error, Warning, Display, Log, Verbose, and VeryVerbose
Log will only go to the log file unless -verbose is passed on the command line
reduced some of the output from UAT to be Log only
[CL 2631062 by Peter Sauerbrei in Main branch]
* Remove ErrorReporter.Error, replace with AutomationException with Error Code.
* Move ErrorCodes to AutomationException.
* Don't return exit codes. Solely rely on exceptions to propagate exit codes.
* Remove MainProc delegate
* Remove setting of Environment.ExitCode as it is ignored when main returns an int.
* ShutdownLogging is nothrow, as all exceptions would be ignored anyway.
* Wrap all shutdown steps so further ones get a chance to run.
* Move HostPlatform.Initialize into the global try/catch block
#codereview:ben.marsh
[CL 2605826 by Wes Hunt in Main branch]
* Allows us to use built-in Trace providers (console, file, etc) directly and still use our custom formatting.
* Fat comments explaining why Trace.WriteXXX functions should not be used directly in our system.
* Fixes thread safety by using Trace.WriteXXX under the hood after formatting, which uses a global lock (except on Mono, where a bug appears to be preventing this. Simulating the call on that platform).
* No need for TraceEvent overloads, which saves us the extra parameter cruft.
* Removed non-varargs overloads of Log functions (technically a bit slower, but these are already small messages).
* No longer needed VerbosityFilter and ConsoleListener classes.
* Avoid calling GetSource() if we aren't outputting the source.
* Avoid formatting the string if it won't pass the verbosity level.
* Consolidated all of UAT and UBT options into this class, so they could fully share the implementation.
UBT BuildConfiguration.cs
* Added LogFilename (and --log=<file> arg) that enables logging to a file.
* Added static ctor guard that asserts if someone tries to read a config before we have loaded config files and parsed config-override commandlines. It's a poor man's hack, but better than nothing!
UBT UEBuildConfiguration.cs
* Same static ctor guard as above.
UBT UnrealBuildTools.cs (initialization refactoring)
* In general I tried to de-mystify some of the rationale behind our startup code via fat comments.
* Broke main into 3 stages:
1. "early code" that should not try to read a config value.
* Very little code here. Mostly setting the current directory.
* Does an early init of logging to ensure logging is around, but config values won't be ready.
2. "Init Configuration code" that loads config files and parses command lines that may override them.
* I isolated two locations in startup that parsed long sets of switches and moved ones that trivially affected BuildConfiguration and UEBuildConfiguration in here. Those two locations seemed to have mostly copies of the same switches, indicating serious param parsing issues at some point in time.
* This allows switches to override config files more easily than the patchwork of re-parsing that was currently used (particularly for -verbose).
* I did a cursory examination of later code that indicated this double (actually, triple) parsing was no longer necessary with the refactors above. Any insight into why things may have ended up this way would be helpful.
3. "Post Init code" that is actually the meat of UBT.
* I left this code largely untouched.
* Removed 2 of 3 different command line logging statements.
* Removed two redundant parses of config overrides (ParseBuildConfigurationFlags).
* Guarded all of main in a try/catch block to ensure no exceptions can leak from UBT without returning a valid error code. It ALMOST already did this, but only covered the part surrounded by the Mutex.
* There was a perplexing bit that redundantly called XmlConfigLoader.Reset<> (line 683) that I struggled to understand. It turns out UEBuildConfiguration was sensitive to the current directory being set before files were loaded, and the old code called XmlConfigLoader.Init() super early, which required it to be called again after the current directory was set (see UEBuldConfiguration.UEThirdPartySourceDirectory for the cause). After my changes, I verified as best I could that these calls are no longer needed and removed them.
XmlConfigLoader.cs
* Add support for Properties in XmlConfigLoader.
AutomationTool Program.cs
* Guard logging shutdown code in try/finally so it can't be missed.
AutomationTool Log.cs
* Uses new logging system from UBT
* Removed unnecessary classes (VerbosityFilter, AutomationConsoleTraceListener, and AutomationFileTraceListener)
* Console trace logic is handled by UBT code now, moved UTF8Output handling to InitLogging.
* A custom TraceListener for file logging was unnecessary.
* Logic to handle creating the log file and retry loops was move into InitLogging, and the result passed to a regular TextFileTraceListener.
* Logic to handle copying the log on shutdown was moved to a ShutdownLogging function.
#codereview:robert.manuszewski,michael.trepka,kellan.carr
[CL 2526245 by Wes Hunt in Main branch]
Added an error code for missing ue4game binaries when packaging a non code project.
Added a check in main frame actions to determine whether we display a dialog with the error message in it.
Fix for incorrect IOS stub path being logged
Fix for incorrect Android apk path being generated for non ue4game renames in ::GetFinalApkName
[CL 2427458 by Terence Burns in Main branch]
