Files
UnrealEngineUWP/Engine/Source/Programs/UnrealBuildTool/Configuration/UEBuildModule.cs
Ben Marsh 75fa4e9e6d Copying //UE4/Dev-Core to //UE4/Dev-Main (Source: //UE4/Dev-Core @ 3386108)
#rb none
#lockdown Nick.Penwarden

==========================
MAJOR FEATURES + CHANGES
==========================

Change 3345860 on 2017/03/14 by Daniel.Lamb

	Fixed crash when building DLC

	#test Cook paragon.

Change 3347324 on 2017/03/15 by Gil.Gribb

	UE4 - Removed old code relating to FAsyncArchive, FAsyncIOSubsystemBase and package level compression. The editor now uses the lowest levels on the new async IO scheme.

Change 3347331 on 2017/03/15 by Robert.Manuszewski

	Fix for a crash caused by GC killing BP class (due to no strong references) but its CDO is being kept alive because it was in the same cluster as the class and was not marked as pending kill.

	#jira UE-42732

Change 3347371 on 2017/03/15 by Graeme.Thornton

	Fix for runtime asset cache not invalidating files with an outdated version number

Change 3349161 on 2017/03/16 by Steve.Robb

	Generated UFUNCTION FNames no longer exported.
	Misc refactors of code generation.

Change 3349167 on 2017/03/16 by Steve.Robb

	Unused TBoolConstant removed (the more general TIntegralConstant should be used instead).

Change 3349274 on 2017/03/16 by Gil.Gribb

	UE4 - Fix loading a package that is already loaded.

Change 3349534 on 2017/03/16 by Ben.Marsh

	UBT: Check that the SN-DBS service is running before attempting to use it.

Change 3349612 on 2017/03/16 by Gil.Gribb

	UE4 - Increased estimate of summary size.

Change 3350021 on 2017/03/16 by Gil.Gribb

	UE4 - Fixed crash in signature checks when mounting pak files.

Change 3350052 on 2017/03/16 by Ben.Marsh

	Remove invalid characters from macro names before passing as macro values. Prevents compile errors for projects which have apostrophes in the name.

Change 3350360 on 2017/03/16 by Ben.Marsh

	UAT: Fix non-threadsafe access of ExeToTimeInMs when spawning external processes.

Change 3351670 on 2017/03/17 by Ben.Marsh

	UBT: Ignore all default libraries when creating import libs. Sometimes #pragma comment(lib, ...) directives can add force additional libraries onto the linker/librarian command line. We don't want or need these included when generating import libraries, but they can cause errors due to search paths not being able to find them.

Change 3352289 on 2017/03/17 by Ben.Marsh

	Fix issues working with > 2GB archives caused by truncation of the return value from FArchive::Tell() down to 32-bits.

Change 3352390 on 2017/03/17 by Ben.Marsh

	Remove unused/out of date binaries for CrashReporter.

Change 3352392 on 2017/03/17 by Ben.Marsh

	Remove UnrealDocTool binaries. This is distributed through a Visual Studio plugin now.

Change 3352410 on 2017/03/17 by Ben.Marsh

	Remove P4ChangeReporter. I don't believe this is used any more.

Change 3352450 on 2017/03/17 by Ben.Marsh

	Disable including CrashReporter by default when packaging projects. This is only useful with a CrashReporter backend set up, which only usually applies to Epic internal projects.

Change 3352455 on 2017/03/17 by Ben.Marsh

	Remove RegisterPII and TranslatedWordsCountEstimator executables. Don't believe these are used any more.

Change 3352940 on 2017/03/17 by Wes.Hunt

	Update CRP to not send Slack queue size updates unless the waiting time is greater  than 1 minute.
	#codereview: jin.zhang

Change 3353658 on 2017/03/20 by Steve.Robb

	Fix for crash when importing a BP which has a populated TMap with an enum class key.

Change 3354056 on 2017/03/20 by Steve.Robb

	TAssetPtr<T> can now be constructed from a nullptr without a full definition of T.

Change 3356111 on 2017/03/21 by Graeme.Thornton

	Fix for UE-34131
	 - Support double and fname stat types in UFE stat export to CSV

	#jira UE-34131

Change 3358584 on 2017/03/22 by Daniel.Lamb

	Fixed the garbage collection keep flags when cleaning the sandbox for iterative cooking.

	#test Cook shootergame

Change 3360379 on 2017/03/23 by Gil.Gribb

	UE4 - Avoid adding a linker annotation if it actually hasn't changed. Improves ConditionalBeginDestroy performance.

Change 3360623 on 2017/03/23 by Gil.Gribb

	UE4 - Change from MarcA to avoid a redudnant removal of PrimitiveComponent from the streaming managers during ConditionalBeginDestroy.

Change 3360627 on 2017/03/23 by Gil.Gribb

	UE4 - Optimized UObject hash tables for speed and space.

Change 3361183 on 2017/03/23 by Gil.Gribb

	UE4 - Fixed change to NotifyPrimitiveDetached so that it works in the editor.

Change 3361906 on 2017/03/23 by Steve.Robb

	Fix for a bad hint index when instantiating map property subobjects when the defaults has fewer but non-zero elements.

	#jira UE-43272

Change 3362839 on 2017/03/24 by Gil.Gribb

	UE4 - Fixed hash table lock optimization.

Change 3367348 on 2017/03/28 by Robert.Manuszewski

	Making sure streamed-in SoundWaves get added to GC clusters.

Change 3367386 on 2017/03/28 by Ben.Marsh

	EC: Pass the Semaphores property from a build type as a parameter to new build jobs.

Change 3367422 on 2017/03/28 by Ben.Marsh

	EC: Allow limiting the number of scheduled jobs that will be automatically run at a particular time. Each build type can have a 'Semaphores' property in the branch settings file, which will be copied to newly created jobs. Before scheduling new jobs, EC is queried for the 'Semaphores' property on any running jobs, and build types with existing semaphores will be skipped. Does not prevent jobs from being run manually.

Change 3367469 on 2017/03/28 by Ben.Marsh

	EC: Prevent multiple incremental jobs running at once.

Change 3367640 on 2017/03/28 by Ben.Marsh

	Plugins: Add an optional EngineVersion field back into the plugin descriptor. If set, the engine will warn if the plugin is not compatible with the current engine version. Plugins will set this field by default when packaging; pass -Unversioned to override.

Change 3367836 on 2017/03/28 by Uriel.Doyon

	Improved handled of references in the streaming manager

Change 3369354 on 2017/03/29 by Graeme.Thornton

	Added AES encrypt/decrypt functions that take a byte array for the key

Change 3369804 on 2017/03/29 by Ben.Marsh

	Remove incorrect "EngineVersion" settings from plugin descriptors.

Change 3370462 on 2017/03/29 by Ben.Marsh

	Editor: Install Visual Studio 2017 by default, instead of Visual Studio 2015. Changed to use ExecElevatedProcess() to prevent installer failing to run if the current user is not already an administrator.

	#jira UE-43467

Change 3371598 on 2017/03/30 by Ben.Marsh

	UBT: Fix message for missing toolchain in VS2017.

Change 3372827 on 2017/03/30 by Ben.Marsh

	BuildGraph: Output an error at the end of each step if any previous build products have been modified.

Change 3372947 on 2017/03/30 by Ben.Marsh

	[Merge] Always add the host editor platform as supported in an installed build. Not doing so prevents the build platform being registered in UBT, which prevents doing any platform-specific staging operations in UAT.

Change 3372958 on 2017/03/30 by Ben.Marsh

	[Merge] Simplify log output for cooks. Suppress additional timestamps from the editor when running through UAT.

Change 3372981 on 2017/03/30 by Ben.Marsh

	[Merge] Modular game fixes for UAT

	* Store list of executable names from the receipts instead of generating them from Target/Platform/Config/Architecture combination
	* Get full list of staged executables from receipts instead of assuming only non-code projects are in Engine
	* Always pass short project name as Bootstrap argument, so that modular game exe knows which project to start

Change 3373024 on 2017/03/30 by Ben.Marsh

	[Merge] Add an option to UAT (-CookOutputDir=...) and the cooker (-OutputDir=...) which allows overriding the output directory for cooked files, and fix situations where the directory becomes too deep.

Change 3373041 on 2017/03/30 by Ben.Marsh

	[Merge] Added UAT script to replace assets with another source
	Renamed ReplaceAssetsCommandlet to GenerateAssetsManifest as it now outputs a list of files and has nothing specific about replacing files

Change 3373052 on 2017/03/30 by Ben.Marsh

	[Merge] Changed CopyUsingDistillFileSet command so that it can use a pre-existing manifest file instead of running commandlet

Change 3373092 on 2017/03/30 by Ben.Marsh

	[Merge] Fixed crash attempting to load cooked static mesh in editor

Change 3373112 on 2017/03/30 by Ben.Marsh

	[Merge] Fixed crash caused by loading cooked StaticMesh in editor that didn't have any SourceModels

Change 3373132 on 2017/03/30 by Ben.Marsh

	[Merge] Added Additional Maps that are always cooked to the GenerateDistillFileSetsCommandlet

Change 3373138 on 2017/03/30 by Ben.Marsh

	[Merge] Fixed code issue with playback of cooked SoundCues
	Skip over code using editor only data when editor data has been stripped

Change 3373143 on 2017/03/30 by Ben.Marsh

	[Merge] Fixed crash when attempting to open multiple cooked assets

Change 3373156 on 2017/03/30 by Ben.Marsh

	[Merge] Added commandlet to replace game assets with those from another source (intended for cooked asset replacement)

Change 3373161 on 2017/03/30 by Ben.Marsh

	[Merge] Prevented crash by not attempting to Load Mips again if a package has cooked data

Change 3373168 on 2017/03/30 by Ben.Marsh

	[Merge] Fix output path for DLC pak file, so it can be discovered by the engine and automatically mounted (and to stop it colliding with the main game pak file).

Change 3373204 on 2017/03/30 by Ben.Marsh

	[Merge] Fix crash when switching levels in PIE, due to bulk data already having been discarded for cooked assets. Cooking sets BULKDATA_SingleUse for textures, but PIEing needs to keep bulk data around.

Change 3373209 on 2017/03/30 by Ben.Marsh

	[Merge] Fix missing material in mod editor for cooked assets.

Change 3373388 on 2017/03/30 by Ben.Marsh

	[Merge] Various improvements to the plugin browser and new plugin wizard from Robo Recall.

Change 3374200 on 2017/03/31 by Ben.Marsh

	[Merge] Latest OdinEditor plugin from //Odin/Main, to fix build failures. Re-made change to OdinUnrealEdEngine to remove dependencies on analytics.

Change 3374279 on 2017/03/31 by Ben.Marsh

	PR #3441: Invalid JSON in FeaturePacks (Contributed by projectgheist)

Change 3374331 on 2017/03/31 by Ben.Marsh

	UBT: Disable warning pragmas on Mono; not supported on current compiler.

	#jira UE-43451

Change 3375108 on 2017/03/31 by Ben.Marsh

	Removing another plugin EngineVersion property.

Change 3375126 on 2017/03/31 by Ben.Marsh

	Fix incorrect executable paths being generated for Windows.

Change 3375159 on 2017/03/31 by Graeme.Thornton

	Pak Index Encryption
	 - Added "-encryptindex" option to unrealpak which will encrypt the pak index, making the pak file unreadable without the associated decryption key
	 - Added "-encryptpakindex" option to UAT to force on index encryption
	 - Added "bEncryptPakIndex" setting to project packaging settings so pak encryption can be controlled via the editor

Change 3375197 on 2017/03/31 by Graeme.Thornton

	Enable pak index encryption in shootergame

Change 3375377 on 2017/03/31 by Ben.Marsh

	Add build node to submit updated UnrealPak binaries for Win64, Mac and Linux. Currently has to be run via a custom build on EC, with the target set to "Submit UnrealPak Binaries".

Change 3376418 on 2017/04/03 by Ben.Marsh

	BuildGraph: Always clear the cached node state when running locally without having to manually specify the -ClearHistory argument. The -Resume argument allows the previous behavior of continuing a previous build.

Change 3376447 on 2017/04/03 by Ben.Marsh

	Build: Remove some unused stream settings

Change 3376469 on 2017/04/03 by Ben.Marsh

	Build: Add a customizable field for the script to use for custom builds in every branch.

Change 3376654 on 2017/04/03 by Ben.Marsh

	Add a fatal error message containing the module with an outstanding reference when trying to unload it.

	#jira UE-42423

Change 3376747 on 2017/04/03 by Gil.Gribb

	UE4 - Fixed crash relating to FGenericAsyncReadFileHandle when not using the EDL.

Change 3377173 on 2017/04/03 by Ben.Marsh

	Make sure callstacks are written to stdout following a crash on a background thread.

Change 3377183 on 2017/04/03 by Ben.Marsh

	Removing support for building VS2013 targets. Ability to generate VS2013 project files is still allowed, but unsupported (via the -2013unsupported command line argument).

Change 3377280 on 2017/04/03 by Ben.Marsh

	Build: Post UGS badges for all UE4 development streams, with the project set to $(Branch)/...

Change 3377311 on 2017/04/03 by Ben.Marsh

	Build: Set the 'Semaphores' parameter for any jobs started from a schedule.

Change 3377326 on 2017/04/03 by Ben.Marsh

	UGS: Show badges which match an entire subtree if the project field ends with "...".

Change 3377392 on 2017/04/03 by Ben.Marsh

	Add badges to UE4/Main and UE4/Release streams, and change the names of the builds in development streams to distinguish them.

Change 3377895 on 2017/04/03 by Ben.Marsh

	EC: Send notification emails whenever UAT fails to compile.

Change 3377923 on 2017/04/03 by Ben.Marsh

	Build: Use a different semaphore for the common editors build target to the incremental compile build target.

Change 3378297 on 2017/04/04 by Graeme.Thornton

	Fix incorrect generation of UE_ENGINE_DIRECTORY in UBT

Change 3378301 on 2017/04/04 by Ben.Marsh

	UBT: Try enabling bAdaptiveUnityDisablesPCH by default, to reduce the number of build failures we see due to missing includes.

Change 3378460 on 2017/04/04 by Graeme.Thornton

	Remove dependency preloading system from sync and async loading paths

Change 3378535 on 2017/04/04 by Robert.Manuszewski

	Fix for audio crash when launching Ocean PIE after removing the audio chunk allocation in CL #3347324

	#jira UE-43544

Change 3378575 on 2017/04/04 by Robert.Manuszewski

	Making sure actor clusters are not created in non-cooked builds

	#jira UE-43617
	#jira UE-43614

Change 3378589 on 2017/04/04 by Robert.Manuszewski

	Disabling debug GC cluster logging

	#jira UE-43617

Change 3379118 on 2017/04/04 by Robert.Manuszewski

	Disabling actor clustering by default, keeping it on in Orion and Ocean

Change 3379815 on 2017/04/04 by Ben.Marsh

	Revert change to derive executable names from target receipts. While a better solution than making them up, Android relies on having the base executable names for supporting multiple architectures.

Change 3380811 on 2017/04/05 by Gil.Gribb

	UE4 - Put the special boot order things into baseengine.ini so that licensees and games can add to it.

Change 3383313 on 2017/04/06 by Uriel.Doyon

	Integrated CL 3372436 3372765 3373272 from Dev-Rendering
	#JIRA UE-43669

Change 3383531 on 2017/04/06 by Ben.Marsh

	UGS: Ignore failures when querying whether paths exist. Permissions can cause this folder to fail, even if it will succeed at a parent directory.

Change 3383786 on 2017/04/06 by Ben.Zeigler

	Back out changelist 3382694 and replace with CL #3383757 from bob.tellez:
	Fix memory stomping issue caused by removing a FFortProfileSynchronizeRequest from SynchronizeRequests in UFortRegisteredPlayerInfo::UpdateSynchronizeRequest before SynchronizeProfile had finished executing

Change 3385089 on 2017/04/07 by Gil.Gribb

	UE4 - Critical. Fixed memory leak in pak precacher.

[CL 3386123 by Ben Marsh in Main branch]
2017-04-10 11:00:33 -04:00

830 lines
33 KiB
C#

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.
using System;
using System.Collections.Generic;
using System.Text;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Xml;
using Tools.DotNETCommon.CaselessDictionary;
namespace UnrealBuildTool
{
/// <summary>
/// A unit of code compilation and linking.
/// </summary>
abstract class UEBuildModule
{
/// <summary>
/// The name that uniquely identifies the module.
/// </summary>
public readonly string Name;
/// <summary>
/// The type of module being built. Used to switch between debug/development and precompiled/source configurations.
/// </summary>
public UHTModuleType Type;
/// <summary>
/// The rules for this module
/// </summary>
public ModuleRules Rules;
/// <summary>
/// Path to the module directory
/// </summary>
public readonly DirectoryReference ModuleDirectory;
/// <summary>
/// Is this module allowed to be redistributed.
/// </summary>
private readonly bool? IsRedistributableOverride;
/// <summary>
/// The name of the .Build.cs file this module was created from, if any
/// </summary>
public FileReference RulesFile;
/// <summary>
/// The binary the module will be linked into for the current target. Only set after UEBuildBinary.BindModules is called.
/// </summary>
public UEBuildBinary Binary = null;
/// <summary>
/// Include path for this module's base directory, relative to the Engine/Source directory
/// </summary>
protected string NormalizedModuleIncludePath;
/// <summary>
/// The name of the _API define for this module
/// </summary>
protected readonly string ModuleApiDefine;
/// <summary>
/// Set of all the public definitions
/// </summary>
protected readonly HashSet<string> PublicDefinitions;
/// <summary>
/// Set of all public include paths
/// </summary>
protected readonly HashSet<string> PublicIncludePaths;
/// <summary>
/// Set of all private include paths
/// </summary>
protected readonly HashSet<string> PrivateIncludePaths;
/// <summary>
/// Set of all system include paths
/// </summary>
protected readonly HashSet<string> PublicSystemIncludePaths;
/// <summary>
/// Set of all public library paths
/// </summary>
protected readonly HashSet<string> PublicLibraryPaths;
/// <summary>
/// Set of all additional libraries
/// </summary>
protected readonly HashSet<string> PublicAdditionalLibraries;
/// <summary>
/// Set of additional frameworks
/// </summary>
protected readonly HashSet<string> PublicFrameworks;
/// <summary>
///
/// </summary>
protected readonly HashSet<string> PublicWeakFrameworks;
/// <summary>
///
/// </summary>
protected readonly HashSet<UEBuildFramework> PublicAdditionalFrameworks;
/// <summary>
///
/// </summary>
protected readonly HashSet<string> PublicAdditionalShadowFiles;
/// <summary>
///
/// </summary>
protected readonly HashSet<UEBuildBundleResource> PublicAdditionalBundleResources;
/// <summary>
/// Names of modules with header files that this module's public interface needs access to.
/// </summary>
protected List<UEBuildModule> PublicIncludePathModules;
/// <summary>
/// Names of modules that this module's public interface depends on.
/// </summary>
protected List<UEBuildModule> PublicDependencyModules;
/// <summary>
/// Names of DLLs that this module should delay load
/// </summary>
protected HashSet<string> PublicDelayLoadDLLs;
/// <summary>
/// Names of modules with header files that this module's private implementation needs access to.
/// </summary>
protected List<UEBuildModule> PrivateIncludePathModules;
/// <summary>
/// Names of modules that this module's private implementation depends on.
/// </summary>
protected List<UEBuildModule> PrivateDependencyModules;
/// <summary>
/// Extra modules this module may require at run time
/// </summary>
protected List<UEBuildModule> DynamicallyLoadedModules;
/// <summary>
/// Extra modules this module may require at run time, that are on behalf of another platform (i.e. shader formats and the like)
/// </summary>
protected List<UEBuildModule> PlatformSpecificDynamicallyLoadedModules;
/// <summary>
/// Files which this module depends on at runtime.
/// </summary>
public RuntimeDependencyList RuntimeDependencies;
/// <summary>
/// Set of all whitelisted restricted folder references
/// </summary>
private readonly HashSet<DirectoryReference> WhitelistRestrictedFolders;
/// <summary>
/// Constructor
/// </summary>
/// <param name="InName">Name of the module</param>
/// <param name="InType">Type of the module, for UHT</param>
/// <param name="InModuleDirectory">Base directory for the module</param>
/// <param name="InRules">Rules for this module</param>
/// <param name="InRulesFile">Path to the rules file</param>
public UEBuildModule(string InName, UHTModuleType InType, DirectoryReference InModuleDirectory, ModuleRules InRules, FileReference InRulesFile)
{
Name = InName;
Type = InType;
ModuleDirectory = InModuleDirectory;
Rules = InRules;
RulesFile = InRulesFile;
NormalizedModuleIncludePath = Utils.CleanDirectorySeparators(ModuleDirectory.MakeRelativeTo(UnrealBuildTool.EngineSourceDirectory), '/');
ModuleApiDefine = Name.ToUpperInvariant() + "_API";
PublicDefinitions = HashSetFromOptionalEnumerableStringParameter(InRules.Definitions);
PublicIncludePaths = HashSetFromOptionalEnumerableStringParameter(InRules.PublicIncludePaths);
PublicSystemIncludePaths = HashSetFromOptionalEnumerableStringParameter(InRules.PublicSystemIncludePaths);
PublicLibraryPaths = HashSetFromOptionalEnumerableStringParameter(InRules.PublicLibraryPaths);
PublicAdditionalLibraries = HashSetFromOptionalEnumerableStringParameter(InRules.PublicAdditionalLibraries);
PublicFrameworks = HashSetFromOptionalEnumerableStringParameter(InRules.PublicFrameworks);
PublicWeakFrameworks = HashSetFromOptionalEnumerableStringParameter(InRules.PublicWeakFrameworks);
PublicAdditionalFrameworks = InRules.PublicAdditionalFrameworks == null ? new HashSet<UEBuildFramework>() : new HashSet<UEBuildFramework>(InRules.PublicAdditionalFrameworks);
PublicAdditionalShadowFiles = HashSetFromOptionalEnumerableStringParameter(InRules.PublicAdditionalShadowFiles);
PublicAdditionalBundleResources = InRules.AdditionalBundleResources == null ? new HashSet<UEBuildBundleResource>() : new HashSet<UEBuildBundleResource>(InRules.AdditionalBundleResources);
PublicDelayLoadDLLs = HashSetFromOptionalEnumerableStringParameter(InRules.PublicDelayLoadDLLs);
PrivateIncludePaths = HashSetFromOptionalEnumerableStringParameter(InRules.PrivateIncludePaths);
RuntimeDependencies = (InRules.RuntimeDependencies == null) ? new RuntimeDependencyList() : new RuntimeDependencyList(InRules.RuntimeDependencies);
IsRedistributableOverride = InRules.IsRedistributableOverride;
WhitelistRestrictedFolders = new HashSet<DirectoryReference>(InRules.WhitelistRestrictedFolders.Select(x => DirectoryReference.Combine(ModuleDirectory, x)));
}
/// <summary>
/// Returns a list of this module's dependencies.
/// </summary>
/// <returns>An enumerable containing the dependencies of the module.</returns>
public HashSet<UEBuildModule> GetDependencies(bool bWithIncludePathModules, bool bWithDynamicallyLoadedModules)
{
HashSet<UEBuildModule> Modules = new HashSet<UEBuildModule>();
Modules.UnionWith(PublicDependencyModules);
Modules.UnionWith(PrivateDependencyModules);
if(bWithIncludePathModules)
{
Modules.UnionWith(PublicIncludePathModules);
Modules.UnionWith(PrivateIncludePathModules);
}
if(bWithDynamicallyLoadedModules)
{
Modules.UnionWith(DynamicallyLoadedModules);
Modules.UnionWith(PlatformSpecificDynamicallyLoadedModules);
}
return Modules;
}
/// <summary>
/// Returns a list of this module's immediate dependencies.
/// </summary>
/// <returns>An enumerable containing the dependencies of the module.</returns>
public IEnumerable<UEBuildModule> GetDirectDependencyModules()
{
return PublicDependencyModules.Concat(PrivateDependencyModules).Concat(DynamicallyLoadedModules).Concat(PlatformSpecificDynamicallyLoadedModules);
}
/// <summary>
/// Converts an optional string list parameter to a well-defined hash set.
/// </summary>
protected static HashSet<string> HashSetFromOptionalEnumerableStringParameter(IEnumerable<string> InEnumerableStrings)
{
return InEnumerableStrings == null ? new HashSet<string>() : new HashSet<string>(InEnumerableStrings);
}
/// <summary>
/// Determines whether this module has a circular dependency on the given module
/// </summary>
public bool HasCircularDependencyOn(string ModuleName)
{
return Rules.CircularlyReferencedDependentModules.Contains(ModuleName);
}
/// <summary>
/// Enumerates additional build products which may be produced by this module. Some platforms (eg. Mac, Linux) can link directly against .so/.dylibs, but they
/// are also copied to the output folder by the toolchain.
/// </summary>
/// <param name="Libraries">List to which libraries required by this module are added</param>
/// <param name="BundleResources">List of bundle resources required by this module</param>
public void GatherAdditionalResources(List<string> Libraries, List<UEBuildBundleResource> BundleResources)
{
Libraries.AddRange(PublicAdditionalLibraries);
BundleResources.AddRange(PublicAdditionalBundleResources);
}
/// <summary>
/// Determines the distribution level of a module based on its directory and includes.
/// </summary>
/// <param name="ProjectDir">The project directory, if available</param>
/// <returns>Map of the restricted folder types to the first found instance</returns>
public Dictionary<RestrictedFolder, DirectoryReference> FindRestrictedFolderReferences(DirectoryReference ProjectDir)
{
Dictionary<RestrictedFolder, DirectoryReference> References = new Dictionary<RestrictedFolder, DirectoryReference>();
if (!Rules.bOutputPubliclyDistributable)
{
// Find all the directories that this module references
HashSet<DirectoryReference> ReferencedDirs = new HashSet<DirectoryReference>();
GetReferencedDirectories(ReferencedDirs);
// Remove all the whitelisted folders
ReferencedDirs.ExceptWith(WhitelistRestrictedFolders);
ReferencedDirs.ExceptWith(PublicDependencyModules.SelectMany(x => x.WhitelistRestrictedFolders));
ReferencedDirs.ExceptWith(PrivateDependencyModules.SelectMany(x => x.WhitelistRestrictedFolders));
// Add flags for each of them
foreach(DirectoryReference ReferencedDir in ReferencedDirs)
{
// Find the base directory containing this reference
DirectoryReference BaseDir;
if(ReferencedDir.IsUnderDirectory(UnrealBuildTool.EngineDirectory))
{
BaseDir = UnrealBuildTool.EngineDirectory;
}
else if(ProjectDir != null && ReferencedDir.IsUnderDirectory(ProjectDir))
{
BaseDir = ProjectDir;
}
else
{
continue;
}
// Add references to each of the restricted folders
List<RestrictedFolder> Folders = RestrictedFolders.FindRestrictedFolders(BaseDir, ReferencedDir);
foreach(RestrictedFolder Folder in Folders)
{
if(!References.ContainsKey(Folder))
{
References.Add(Folder, ReferencedDir);
}
}
}
}
return References;
}
/// <summary>
/// Finds all the directories that this folder references when building
/// </summary>
/// <param name="Directories">Set of directories to add to</param>
protected virtual void GetReferencedDirectories(HashSet<DirectoryReference> Directories)
{
Directories.Add(ModuleDirectory);
foreach(string PublicIncludePath in PublicIncludePaths)
{
Directories.Add(new DirectoryReference(PublicIncludePath));
}
foreach(string PrivateIncludePath in PrivateIncludePaths)
{
Directories.Add(new DirectoryReference(PrivateIncludePath));
}
foreach(string PublicSystemIncludePath in PublicSystemIncludePaths)
{
Directories.Add(new DirectoryReference(PublicSystemIncludePath));
}
foreach(string PublicLibraryPath in PublicLibraryPaths)
{
Directories.Add(new DirectoryReference(PublicLibraryPath));
}
}
/// <summary>
/// Find all the modules which affect the public compile environment. Searches through
/// </summary>
/// <param name="Modules"></param>
/// <param name="ModuleToIncludePathsOnlyFlag"></param>
protected void FindModulesInPublicCompileEnvironment(List<UEBuildModule> Modules, Dictionary<UEBuildModule, bool> ModuleToIncludePathsOnlyFlag)
{
//
bool bModuleIncludePathsOnly;
if (!ModuleToIncludePathsOnlyFlag.TryGetValue(this, out bModuleIncludePathsOnly))
{
Modules.Add(this);
}
else if (!bModuleIncludePathsOnly)
{
return;
}
ModuleToIncludePathsOnlyFlag[this] = false;
foreach (UEBuildModule DependencyModule in PublicDependencyModules)
{
DependencyModule.FindModulesInPublicCompileEnvironment(Modules, ModuleToIncludePathsOnlyFlag);
}
// Now add an include paths from modules with header files that we need access to, but won't necessarily be importing
foreach (UEBuildModule IncludePathModule in PublicIncludePathModules)
{
IncludePathModule.FindIncludePathModulesInPublicCompileEnvironment(Modules, ModuleToIncludePathsOnlyFlag);
}
}
/// <summary>
/// Find all the modules which affect the public compile environment. Searches through
/// </summary>
/// <param name="Modules"></param>
/// <param name="ModuleToIncludePathsOnlyFlag"></param>
protected void FindIncludePathModulesInPublicCompileEnvironment(List<UEBuildModule> Modules, Dictionary<UEBuildModule, bool> ModuleToIncludePathsOnlyFlag)
{
if (!ModuleToIncludePathsOnlyFlag.ContainsKey(this))
{
// Add this module to the list
Modules.Add(this);
ModuleToIncludePathsOnlyFlag.Add(this, true);
// Include any of its public include path modules in the compile environment too
foreach (UEBuildModule IncludePathModule in PublicIncludePathModules)
{
IncludePathModule.FindIncludePathModulesInPublicCompileEnvironment(Modules, ModuleToIncludePathsOnlyFlag);
}
}
}
/// <summary>
/// Sets up the environment for compiling any module that includes the public interface of this module.
/// </summary>
public void AddModuleToCompileEnvironment(
UEBuildBinary SourceBinary,
bool bIncludePathsOnly,
HashSet<string> IncludePaths,
HashSet<string> SystemIncludePaths,
List<string> Definitions,
List<UEBuildFramework> AdditionalFrameworks
)
{
// Add this module's public include paths and definitions.
AddIncludePathsWithChecks(IncludePaths, PublicIncludePaths);
AddIncludePathsWithChecks(SystemIncludePaths, PublicSystemIncludePaths);
Definitions.AddRange(PublicDefinitions);
// If this module is being built into a DLL or EXE, set up an IMPORTS or EXPORTS definition for it.
if(SourceBinary == null)
{
// No source binary means a shared PCH, so always import all symbols. It's possible that an include path module now may be a imported module for the shared PCH consumer.
if(!bIncludePathsOnly)
{
if(Binary == null || !Binary.Config.bAllowExports)
{
Definitions.Add(ModuleApiDefine + "=");
}
else
{
Definitions.Add(ModuleApiDefine + "=DLLIMPORT");
}
}
}
else if (Binary == null)
{
// If we're referencing include paths for a module that's not being built, we don't actually need to import anything from it, but we need to avoid barfing when
// the compiler encounters an _API define. We also want to avoid changing the compile environment in cases where the module happens to be compiled because it's a dependency
// of something else, which cause a fall-through to the code below and set up an empty _API define.
if (bIncludePathsOnly)
{
Log.TraceVerbose("{0}: Include paths only for {1} (no binary)", SourceBinary.Config.OutputFilePaths[0].GetFileNameWithoutExtension(), Name);
Definitions.Add(ModuleApiDefine + "=");
}
}
else
{
FileReference BinaryPath = Binary.Config.OutputFilePaths[0];
FileReference SourceBinaryPath = SourceBinary.Config.OutputFilePaths[0];
if (ProjectFileGenerator.bGenerateProjectFiles || (Binary.Config.Type == UEBuildBinaryType.StaticLibrary))
{
// When generating IntelliSense files, never add dllimport/dllexport specifiers as it
// simply confuses the compiler
Definitions.Add(ModuleApiDefine + "=");
}
else if (Binary == SourceBinary)
{
if (Binary.Config.bAllowExports)
{
Log.TraceVerbose("{0}: Exporting {1} from {2}", SourceBinaryPath.GetFileNameWithoutExtension(), Name, BinaryPath.GetFileNameWithoutExtension());
Definitions.Add(ModuleApiDefine + "=DLLEXPORT");
}
else
{
Log.TraceVerbose("{0}: Not importing/exporting {1} (binary: {2})", SourceBinaryPath.GetFileNameWithoutExtension(), Name, BinaryPath.GetFileNameWithoutExtension());
Definitions.Add(ModuleApiDefine + "=");
}
}
else
{
// @todo SharedPCH: Public headers included from modules that are not importing the module of that public header, seems invalid.
// Those public headers have no business having APIs in them. OnlineSubsystem has some public headers like this. Without changing
// this, we need to suppress warnings at compile time.
if (bIncludePathsOnly)
{
Log.TraceVerbose("{0}: Include paths only for {1} (binary: {2})", SourceBinaryPath.GetFileNameWithoutExtension(), Name, BinaryPath.GetFileNameWithoutExtension());
Definitions.Add(ModuleApiDefine + "=");
}
else if (Binary.Config.bAllowExports)
{
Log.TraceVerbose("{0}: Importing {1} from {2}", SourceBinaryPath.GetFileNameWithoutExtension(), Name, BinaryPath.GetFileNameWithoutExtension());
Definitions.Add(ModuleApiDefine + "=DLLIMPORT");
}
else
{
Log.TraceVerbose("{0}: Not importing/exporting {1} (binary: {2})", SourceBinaryPath.GetFileNameWithoutExtension(), Name, BinaryPath.GetFileNameWithoutExtension());
Definitions.Add(ModuleApiDefine + "=");
}
}
}
// Add the module's directory to the include path, so we can root #includes to it
IncludePaths.Add(NormalizedModuleIncludePath);
// Add the additional frameworks so that the compiler can know about their #include paths
AdditionalFrameworks.AddRange(PublicAdditionalFrameworks);
// Remember the module so we can refer to it when needed
foreach (UEBuildFramework Framework in PublicAdditionalFrameworks)
{
Framework.OwningModule = this;
}
}
static Regex VCMacroRegex = new Regex(@"\$\([A-Za-z0-9_]+\)");
/// <summary>
/// Checks if path contains a VC macro
/// </summary>
protected bool DoesPathContainVCMacro(string Path)
{
return VCMacroRegex.IsMatch(Path);
}
/// <summary>
/// Adds PathsToAdd to IncludePaths, performing path normalization and ignoring duplicates.
/// </summary>
protected void AddIncludePathsWithChecks(HashSet<string> IncludePaths, HashSet<string> PathsToAdd)
{
if (ProjectFileGenerator.bGenerateProjectFiles)
{
// Extra checks are switched off for IntelliSense generation as they provide
// no additional value and cause performance impact.
IncludePaths.UnionWith(PathsToAdd);
}
else
{
foreach (string Path in PathsToAdd)
{
string NormalizedPath = Path.TrimEnd('/');
// If path doesn't exist, it may contain VC macro (which is passed directly to and expanded by compiler).
if (Directory.Exists(NormalizedPath) || DoesPathContainVCMacro(NormalizedPath))
{
IncludePaths.Add(NormalizedPath);
}
}
}
}
/// <summary>
/// Sets up the environment for compiling this module.
/// </summary>
protected virtual void SetupPrivateCompileEnvironment(
HashSet<string> IncludePaths,
HashSet<string> SystemIncludePaths,
List<string> Definitions,
List<UEBuildFramework> AdditionalFrameworks
)
{
HashSet<UEBuildModule> VisitedModules = new HashSet<UEBuildModule>();
if (this.Type.IsGameModule())
{
Definitions.Add("DEPRECATED_FORGAME=DEPRECATED");
}
// Add this module's private include paths and definitions.
AddIncludePathsWithChecks(IncludePaths, PrivateIncludePaths);
// Find all the modules that are part of the public compile environment for this module.
List<UEBuildModule> Modules = new List<UEBuildModule>();
Dictionary<UEBuildModule, bool> ModuleToIncludePathsOnlyFlag = new Dictionary<UEBuildModule, bool>();
FindModulesInPublicCompileEnvironment(Modules, ModuleToIncludePathsOnlyFlag);
// Add in all the modules that are private dependencies
foreach (UEBuildModule DependencyModule in PrivateDependencyModules)
{
DependencyModule.FindModulesInPublicCompileEnvironment(Modules, ModuleToIncludePathsOnlyFlag);
}
// And finally add in all the modules that are include path only dependencies
foreach (UEBuildModule IncludePathModule in PrivateIncludePathModules)
{
IncludePathModule.FindIncludePathModulesInPublicCompileEnvironment(Modules, ModuleToIncludePathsOnlyFlag);
}
// Now set up the compile environment for the modules in the original order that we encountered them
foreach (UEBuildModule Module in Modules)
{
Module.AddModuleToCompileEnvironment(Binary, ModuleToIncludePathsOnlyFlag[Module], IncludePaths, SystemIncludePaths, Definitions, AdditionalFrameworks);
}
}
/// <summary>
/// Sets up the environment for linking any module that includes the public interface of this module.
/// </summary>
protected virtual void SetupPublicLinkEnvironment(
UEBuildBinary SourceBinary,
List<string> LibraryPaths,
List<string> AdditionalLibraries,
List<string> Frameworks,
List<string> WeakFrameworks,
List<UEBuildFramework> AdditionalFrameworks,
List<string> AdditionalShadowFiles,
List<UEBuildBundleResource> AdditionalBundleResources,
List<string> DelayLoadDLLs,
List<UEBuildBinary> BinaryDependencies,
HashSet<UEBuildModule> VisitedModules
)
{
// There may be circular dependencies in compile dependencies, so we need to avoid reentrance.
if (VisitedModules.Add(this))
{
// Add this module's binary to the binary dependencies.
if (Binary != null
&& Binary != SourceBinary
&& !BinaryDependencies.Contains(Binary))
{
BinaryDependencies.Add(Binary);
}
// If this module belongs to a static library that we are not currently building, recursively add the link environment settings for all of its dependencies too.
// Keep doing this until we reach a module that is not part of a static library (or external module, since they have no associated binary).
// Static libraries do not contain the symbols for their dependencies, so we need to recursively gather them to be linked into other binary types.
bool bIsBuildingAStaticLibrary = (SourceBinary != null && SourceBinary.Config.Type == UEBuildBinaryType.StaticLibrary);
bool bIsModuleBinaryAStaticLibrary = (Binary != null && Binary.Config.Type == UEBuildBinaryType.StaticLibrary);
if (!bIsBuildingAStaticLibrary && bIsModuleBinaryAStaticLibrary)
{
// Gather all dependencies and recursively call SetupPublicLinkEnvironmnet
List<UEBuildModule> AllDependencyModules = new List<UEBuildModule>();
AllDependencyModules.AddRange(PrivateDependencyModules);
AllDependencyModules.AddRange(PublicDependencyModules);
foreach (UEBuildModule DependencyModule in AllDependencyModules)
{
bool bIsExternalModule = (DependencyModule as UEBuildModuleExternal != null);
bool bIsInStaticLibrary = (DependencyModule.Binary != null && DependencyModule.Binary.Config.Type == UEBuildBinaryType.StaticLibrary);
if (bIsExternalModule || bIsInStaticLibrary)
{
DependencyModule.SetupPublicLinkEnvironment(SourceBinary, LibraryPaths, AdditionalLibraries, Frameworks, WeakFrameworks,
AdditionalFrameworks, AdditionalShadowFiles, AdditionalBundleResources, DelayLoadDLLs, BinaryDependencies, VisitedModules);
}
}
}
// Add this module's public include library paths and additional libraries.
LibraryPaths.AddRange(PublicLibraryPaths);
AdditionalLibraries.AddRange(PublicAdditionalLibraries);
Frameworks.AddRange(PublicFrameworks);
WeakFrameworks.AddRange(PublicWeakFrameworks);
AdditionalBundleResources.AddRange(PublicAdditionalBundleResources);
// Remember the module so we can refer to it when needed
foreach (UEBuildFramework Framework in PublicAdditionalFrameworks)
{
Framework.OwningModule = this;
}
AdditionalFrameworks.AddRange(PublicAdditionalFrameworks);
AdditionalShadowFiles.AddRange(PublicAdditionalShadowFiles);
DelayLoadDLLs.AddRange(PublicDelayLoadDLLs);
}
}
/// <summary>
/// Sets up the environment for linking this module.
/// </summary>
public virtual void SetupPrivateLinkEnvironment(
UEBuildBinary SourceBinary,
LinkEnvironment LinkEnvironment,
List<UEBuildBinary> BinaryDependencies,
HashSet<UEBuildModule> VisitedModules
)
{
// Allow the module's public dependencies to add library paths and additional libraries to the link environment.
SetupPublicLinkEnvironment(SourceBinary, LinkEnvironment.LibraryPaths, LinkEnvironment.AdditionalLibraries, LinkEnvironment.Frameworks, LinkEnvironment.WeakFrameworks,
LinkEnvironment.AdditionalFrameworks, LinkEnvironment.AdditionalShadowFiles, LinkEnvironment.AdditionalBundleResources, LinkEnvironment.DelayLoadDLLs, BinaryDependencies, VisitedModules);
// Also allow the module's public and private dependencies to modify the link environment.
List<UEBuildModule> AllDependencyModules = new List<UEBuildModule>();
AllDependencyModules.AddRange(PrivateDependencyModules);
AllDependencyModules.AddRange(PublicDependencyModules);
foreach (UEBuildModule DependencyModule in AllDependencyModules)
{
DependencyModule.SetupPublicLinkEnvironment(SourceBinary, LinkEnvironment.LibraryPaths, LinkEnvironment.AdditionalLibraries, LinkEnvironment.Frameworks, LinkEnvironment.WeakFrameworks,
LinkEnvironment.AdditionalFrameworks, LinkEnvironment.AdditionalShadowFiles, LinkEnvironment.AdditionalBundleResources, LinkEnvironment.DelayLoadDLLs, BinaryDependencies, VisitedModules);
}
}
/// <summary>
/// Compiles the module, and returns a list of files output by the compiler.
/// </summary>
public abstract List<FileItem> Compile(ReadOnlyTargetRules Target, UEToolChain ToolChain, CppCompileEnvironment CompileEnvironment, List<PrecompiledHeaderTemplate> SharedPCHModules, ActionGraph ActionGraph);
// Object interface.
public override string ToString()
{
return Name;
}
/// <summary>
/// Finds the modules referenced by this module which have not yet been bound to a binary
/// </summary>
/// <returns>List of unbound modules</returns>
public List<UEBuildModule> GetUnboundReferences()
{
List<UEBuildModule> Modules = new List<UEBuildModule>();
Modules.AddRange(PrivateDependencyModules.Where(x => x.Binary == null));
Modules.AddRange(PublicDependencyModules.Where(x => x.Binary == null));
return Modules;
}
/// <summary>
/// Gets all of the modules referenced by this module
/// </summary>
/// <param name="ReferencedModules">Hash of all referenced modules with their addition index.</param>
/// <param name="IgnoreReferencedModules">Hashset used to ignore modules which are already added to the list</param>
/// <param name="bIncludeDynamicallyLoaded">True if dynamically loaded modules (and all of their dependent modules) should be included.</param>
/// <param name="bForceCircular">True if circular dependencies should be processed</param>
/// <param name="bOnlyDirectDependencies">True to return only this module's direct dependencies</param>
public virtual void GetAllDependencyModules(List<UEBuildModule> ReferencedModules, HashSet<UEBuildModule> IgnoreReferencedModules, bool bIncludeDynamicallyLoaded, bool bForceCircular, bool bOnlyDirectDependencies)
{
}
/// <summary>
/// Gets all of the modules precompiled along with this module
/// </summary>
/// <param name="Modules">Set of all the precompiled modules</param>
public virtual void RecursivelyAddPrecompiledModules(List<UEBuildModule> Modules)
{
}
public delegate UEBuildModule CreateModuleDelegate(string Name);
/// <summary>
/// Creates all the modules required for this target
/// </summary>
public void RecursivelyCreateModules(CreateModuleDelegate CreateModule)
{
// Create all the include path modules. These modules may not be added to the target (and we don't process their dependencies), but they need
// to be created to set up their compile environment.
RecursivelyCreateIncludePathModulesByName(Rules.PublicIncludePathModuleNames, ref PublicIncludePathModules, CreateModule);
RecursivelyCreateIncludePathModulesByName(Rules.PrivateIncludePathModuleNames, ref PrivateIncludePathModules, CreateModule);
// Create all the dependency modules
RecursivelyCreateModulesByName(Rules.PublicDependencyModuleNames, ref PublicDependencyModules, CreateModule);
RecursivelyCreateModulesByName(Rules.PrivateDependencyModuleNames, ref PrivateDependencyModules, CreateModule);
RecursivelyCreateModulesByName(Rules.DynamicallyLoadedModuleNames, ref DynamicallyLoadedModules, CreateModule);
RecursivelyCreateModulesByName(Rules.PlatformSpecificDynamicallyLoadedModuleNames, ref PlatformSpecificDynamicallyLoadedModules, CreateModule);
}
private static void RecursivelyCreateModulesByName(List<string> ModuleNames, ref List<UEBuildModule> Modules, CreateModuleDelegate CreateModule)
{
// Check whether the module list is already set. We set this immediately (via the ref) to avoid infinite recursion.
if (Modules == null)
{
Modules = new List<UEBuildModule>();
foreach (string ModuleName in ModuleNames)
{
UEBuildModule Module = CreateModule(ModuleName);
if (!Modules.Contains(Module))
{
Module.RecursivelyCreateModules(CreateModule);
Modules.Add(Module);
}
}
}
}
private static void RecursivelyCreateIncludePathModulesByName(List<string> ModuleNames, ref List<UEBuildModule> Modules, CreateModuleDelegate CreateModule)
{
// Check whether the module list is already set. We set this immediately (via the ref) to avoid infinite recursion.
if (Modules == null)
{
Modules = new List<UEBuildModule>();
foreach (string ModuleName in ModuleNames)
{
UEBuildModule Module = CreateModule(ModuleName);
RecursivelyCreateIncludePathModulesByName(Module.Rules.PublicIncludePathModuleNames, ref Module.PublicIncludePathModules, CreateModule);
Modules.Add(Module);
}
}
}
/// <summary>
/// Write information about this binary to a JSON file
/// </summary>
/// <param name="Writer">Writer for this binary's data</param>
public virtual void ExportJson(JsonWriter Writer)
{
Writer.WriteValue("Name", Name);
Writer.WriteValue("Type", Type.ToString());
Writer.WriteValue("Directory", ModuleDirectory.FullName);
Writer.WriteValue("Rules", RulesFile.FullName);
Writer.WriteValue("PCHUsage", Rules.PCHUsage.ToString());
if (Rules.PrivatePCHHeaderFile != null)
{
Writer.WriteValue("PrivatePCH", FileReference.Combine(ModuleDirectory, Rules.PrivatePCHHeaderFile).FullName);
}
if (Rules.SharedPCHHeaderFile != null)
{
Writer.WriteValue("SharedPCH", FileReference.Combine(ModuleDirectory, Rules.SharedPCHHeaderFile).FullName);
}
ExportJsonModuleArray(Writer, "PublicDependencyModules", PublicDependencyModules);
ExportJsonModuleArray(Writer, "PublicIncludePathModules", PublicIncludePathModules);
ExportJsonModuleArray(Writer, "PrivateDependencyModules", PrivateDependencyModules);
ExportJsonModuleArray(Writer, "PrivateIncludePathModules", PrivateIncludePathModules);
ExportJsonModuleArray(Writer, "DynamicallyLoadedModules", DynamicallyLoadedModules);
Writer.WriteArrayStart("CircularlyReferencedModules");
foreach(string ModuleName in Rules.CircularlyReferencedDependentModules)
{
Writer.WriteValue(ModuleName);
}
Writer.WriteArrayEnd();
Writer.WriteArrayStart("RuntimeDependencies");
foreach(RuntimeDependency RuntimeDependency in Rules.RuntimeDependencies)
{
Writer.WriteObjectStart();
Writer.WriteValue("Path", RuntimeDependency.Path);
Writer.WriteValue("Type", RuntimeDependency.Type.ToString());
Writer.WriteObjectEnd();
}
Writer.WriteArrayEnd();
}
/// <summary>
/// Write an array of module names to a JSON writer
/// </summary>
/// <param name="Writer">Writer for the array data</param>
/// <param name="ArrayName">Name of the array property</param>
/// <param name="Modules">Sequence of modules to write. May be null.</param>
void ExportJsonModuleArray(JsonWriter Writer, string ArrayName, IEnumerable<UEBuildModule> Modules)
{
Writer.WriteArrayStart(ArrayName);
if (Modules != null)
{
foreach (UEBuildModule Module in Modules)
{
Writer.WriteValue(Module.Name);
}
}
Writer.WriteArrayEnd();
}
};
}