* Allows us to use built-in Trace providers (console, file, etc) directly and still use our custom formatting.
* Fat comments explaining why Trace.WriteXXX functions should not be used directly in our system.
* Fixes thread safety by using Trace.WriteXXX under the hood after formatting, which uses a global lock (except on Mono, where a bug appears to be preventing this. Simulating the call on that platform).
* No need for TraceEvent overloads, which saves us the extra parameter cruft.
* Removed non-varargs overloads of Log functions (technically a bit slower, but these are already small messages).
* No longer needed VerbosityFilter and ConsoleListener classes.
* Avoid calling GetSource() if we aren't outputting the source.
* Avoid formatting the string if it won't pass the verbosity level.
* Consolidated all of UAT and UBT options into this class, so they could fully share the implementation.
UBT BuildConfiguration.cs
* Added LogFilename (and --log=<file> arg) that enables logging to a file.
* Added static ctor guard that asserts if someone tries to read a config before we have loaded config files and parsed config-override commandlines. It's a poor man's hack, but better than nothing!
UBT UEBuildConfiguration.cs
* Same static ctor guard as above.
UBT UnrealBuildTools.cs (initialization refactoring)
* In general I tried to de-mystify some of the rationale behind our startup code via fat comments.
* Broke main into 3 stages:
1. "early code" that should not try to read a config value.
* Very little code here. Mostly setting the current directory.
* Does an early init of logging to ensure logging is around, but config values won't be ready.
2. "Init Configuration code" that loads config files and parses command lines that may override them.
* I isolated two locations in startup that parsed long sets of switches and moved ones that trivially affected BuildConfiguration and UEBuildConfiguration in here. Those two locations seemed to have mostly copies of the same switches, indicating serious param parsing issues at some point in time.
* This allows switches to override config files more easily than the patchwork of re-parsing that was currently used (particularly for -verbose).
* I did a cursory examination of later code that indicated this double (actually, triple) parsing was no longer necessary with the refactors above. Any insight into why things may have ended up this way would be helpful.
3. "Post Init code" that is actually the meat of UBT.
* I left this code largely untouched.
* Removed 2 of 3 different command line logging statements.
* Removed two redundant parses of config overrides (ParseBuildConfigurationFlags).
* Guarded all of main in a try/catch block to ensure no exceptions can leak from UBT without returning a valid error code. It ALMOST already did this, but only covered the part surrounded by the Mutex.
* There was a perplexing bit that redundantly called XmlConfigLoader.Reset<> (line 683) that I struggled to understand. It turns out UEBuildConfiguration was sensitive to the current directory being set before files were loaded, and the old code called XmlConfigLoader.Init() super early, which required it to be called again after the current directory was set (see UEBuldConfiguration.UEThirdPartySourceDirectory for the cause). After my changes, I verified as best I could that these calls are no longer needed and removed them.
XmlConfigLoader.cs
* Add support for Properties in XmlConfigLoader.
AutomationTool Program.cs
* Guard logging shutdown code in try/finally so it can't be missed.
AutomationTool Log.cs
* Uses new logging system from UBT
* Removed unnecessary classes (VerbosityFilter, AutomationConsoleTraceListener, and AutomationFileTraceListener)
* Console trace logic is handled by UBT code now, moved UTF8Output handling to InitLogging.
* A custom TraceListener for file logging was unnecessary.
* Logic to handle creating the log file and retry loops was move into InitLogging, and the result passed to a regular TextFileTraceListener.
* Logic to handle copying the log on shutdown was moved to a ShutdownLogging function.
#codereview:robert.manuszewski,michael.trepka,kellan.carr
[CL 2526245 by Wes Hunt in Main branch]
- Fast UnrealBuildTool startup and dependency checking
- Preflighted, but needs wider testing and debugging. More info to come soon.
#codereview michael.trepka
[CL 2401108 by Mike Fricker in Main branch]
- Collapsed and renamed some of the new UBT settings for fast build iteration
- New 'BuildConfiguration.bUseUBTMakefiles' that replaces the old experimental settings (defaults to off in this changelist.)
- Cleaned up some comments
- No effective changes
[CL 2400071 by Mike Fricker in Main branch]
- Fixed .ini parsing of bools to allow for True, true, TRUE, etc.
#codereview Robert.Manuszewski,niklas.smedberg
[CL 2284471 by Josh Adams in Main branch]
- Enables support for 64-bit compiling under Clang on Windows platform
- Adds initial support for LLVM linker on Windows (WindowsPlatform.bAllowClangLinker) (NOT working yet.)
- Miscellaneous Clang compiling fixes
[CL 2258235 by Mike Fricker in Main branch]
UnrealBuildTool 'Makefiles' allow for very fast iterative builds.
- New BuildConfiguration.xml setting added: "bUseExperimentalFastBuildIteration" (disabled by default)
- Turning this on causes Unreal Build Tool to emit 'UBT Makefiles' for targets when they're built the first time.
- Subsequent builds will load these Makefiles and begin outdatedness checking and build invocation very quickly.
- The caveat is that if source files are added or removed to the project, UBT will need to gather information about those in order for your build to complete successfully.
- Currently, you must run the project file generator after adding/removing source files to tell UBT to re-gather this information.
- Events that can invalidate the 'UBT Makefile':
- Adding/removing .cpp files
- Adding/removing .h files with UObjects
- Adding new UObject types to a file that didn't previously have any
- Changing global build settings (most settings in this file qualify.)
- Changed code that affects how Unreal Header Tool works
- You can force regeneration of the 'UBT Makefile' by passing the '-Gather' argument, or simply regenerating project files
- New command-line parameters added:
- "-Gather": Tells UBT to always perform the gather step (slower but will catch project structural changes)
- "-NoGather": Disables the gather step, unless UBT detects that it must be done. This is the default when bUseExperimentalFastBuildIteration is enabled
- "-GatherOnly": Runs the gather step and saves a UBTMakefile, but doesn't build anything
- "-Assemble": Tells UBT to also assemble build products. This always defaults to enabled
- "-NoAssemble": Tells UBT to skip the assemble step, whether we gathered build products or not
- "-AssembleOnly": Tells UBT to only assemble build products and not to gather, unless UBT determines it must
Other changes:
- UBT now keeps track of which targets it was building in an intermediate file, to help it invalidate cached includes in subsequent runs when the targets are different
- C++ includes are now stored in a class separate from the C++ compile enviroment (for easier serialization)
- The method that UBT uses to find the CoreUObject module timestamp was rewritten
- Various '@todo ubtmake' comments added to tag possible remaining Makefile tasks
- The 'FileItem' class had some member variable comments and code cleaned up, while making it serializable
- Cleaned up the comments and member variables in the "Action" class, while making it serializable
- Some UBT classes are now "serializable". This is because we need to store the data in UBTMakefiles.
- Removed support for Actions to tinker with Stdout and Stderror (was not used for anything)
- Moved PrecompileHeaderEnvironment class to the UEBuildModule.cs source file
- Plugin intermediate include directories are now selected on demand rather than cached early
- Toolchain code for gathering prerequisite headers is now shared in a single function (AddPrerequisiteSourceFile)
- Removed Action.StatusDetailedDescription, was not used for anything
- Removed UEBuildConfiguration.bExcludePlugins, was not used for anything
- Removed ECompilationResult.FailedDueToHeaderChange, was not used for anything
[CL 2254472 by Mike Fricker in Main branch]
#change UBT will check for API version of all UnrealHeaderTool binaries to detect partial syncs
#change Added BuildHostPlatform class for runtime platform abstraction
[CL 2245408 by Robert Manuszewski in Main branch]
- Adds experimental super-fast C++ outdated file checking
- This feature is turned off for now as we continue to test and improve it
- You can try it out by enabling "bUseExperimentalFastDependencyScan" option in your BuildConfiguration.xml
- Here is the basic idea:
- We no longer exhaustively scan all includes and build up a big graph every invocation
- Instead, source files whose build products are missing have their includes scanned asynchronously while being compiled
- The flat list of dependent includes for every outdated C++ is saved into a new cache file ("FlatCPPIncludes.bin")
- On the next run, we quickly load that up and "just know" which files to check timestamps on to determine what is out of date
- Lots of "@todo fastubt" comments were added to UnrealBuildTool for potential performance optimizations and further improvements on this feature.
UnrealBuildTool: Determination of which modules have UObjects is now faster
- We now cache which modules have UObjects and load those for the next session
UnrealBuildTool: Module "shared" precompiled header determination is now much faster
- We no longer scan all C++ includes for a module to figure out which "shared" PCH to use
- Instead, we use the module dependencies specified in the module's *.Build.cs file
- For example, if your module depends on "Engine" and "UnrealEd", we choose "UnrealEd"'s shared PCH
Other UBT optimizations:
- Reduced calls to string formatting functions when setting up API definitions for all modules
- Added new performance diagnostics when bPrintPerformanceInfo is enabled in BuildConfiguration.xml
- We no longer check for "external" headers when scanning includes (this code didn't work at all)
- Optimized CleanDirectorySeparators() utility function to avoid string copies
Fixed UnrealBuildTool not saving DependencyCache under a platform-named folder
[CL 2238266 by Mike Fricker in Main branch]
By either manually configuring distcc & DMUCS as per their documentation, or using the DistCode Xcode plugin compilation can be distributed amongst networked Macs.
All Macs must have the same developer tools installed & the same distcc/DMUCS/DistCode and be on the same local network.
This works for compiling for Mac & iOS and it should be possible to configure distcc & DMUCS manually for use on build farms for remote building from Windows.
Linux could also use this for local builds or remote build farms - but I'm not in a position to test this.
[CL 2238171 by Mark Satterthwaite in Main branch]
If UBT is building a non-editor configuration it throws an error if the produced binaries has non-redistributable dependencies.
User can avoid throwing error and change it to warning at the end with flag BuildConfiguration.bBreakBuildOnLicenseViolation = false.
In general module to be redistributable it must not be neither Development nor Editor type. This behavior can be overridden though. One have to just assign true or false to IsRedistributableOverride field of ModuleRules in <ModuleName>.Build.cs file.
UBT will not check for the dependencies also if the module being built is non-redistributable (e.g. some internal programs).
#ttp 335755
#codereview Robert.Manuszewski
[CL 2225366 by Jaroslaw Palczynski in Main branch]
- XSD driven IntelliSense
- defaults read from code (default XML file will be regenerated on UBT startup if is different than it should be)
- configurable classes/fields have to be now annotated with XmlConfig and XmlConfigField attributes
#codereview Robert.Manuszewski
[CL 2095741 by Jaroslaw Palczynski in Main branch]
