Files
UnrealEngineUWP/Engine/Source/Developer/HotReload/Private/HotReload.cpp
Ben Marsh 30f891786a Copying //UE4/Dev-Core to //UE4/Dev-Main (Source: //UE4/Dev-Core @ 3847469)
#lockdown Nick.Penwarden
#rb none

============================
  MAJOR FEATURES & CHANGES
============================

Change 3805828 by Gil.Gribb

	UE4 - Fixed a bug in the lock free stalling task queue and adjusted a comment. The code is not current used, so this is not actually change the way the code works.

Change 3806784 by Ben.Marsh

	UAT: Remove code to compile UBT when using UE4Build. It should already be compiled as a dependency of UAT.

Change 3807549 by Graeme.Thornton

	Add a cook timer around VerifyCanCookPackage. A licensee reports this taking a lot of time so it'll be good to account for it.

Change 3807727 by Graeme.Thornton

	Unhide the text asset format experimental editor option

Change 3807746 by Josh.Engebretson

	Remove WER from iOS platform

Change 3807928 by Robert.Manuszewski

	When async loading, GC Clusters will be created after packages have been processed to avoid situations where some of the objects that are being added to a cluster haven't been fully loaded yet

Change 3808221 by Steve.Robb

	GitHub #4307 - Made GetModulePtr() thread safe by not using GetModule()

	^ I'm not convinced by how much thread-safer this is really, but it's tidier anyway.

Change 3809233 by Graeme.Thornton

	TBA: Misc changes to text asset commandlet
	 - Rename mode to "loadsave"
	 - Add -outputFormat option which can be assigned "text" or "binary"
	 - When saving binary, use a differentiated filename so that source assets aren't overwritten

Change 3809518 by Ben.Marsh

	Remove the outdated UnrealSync automation script.

Change 3809643 by Steve.Robb

	GitHub #4277 : fix bug; FMath::FormatIntToHumanReadable 3rd comma and negative value

	#jira UE-53037

Change 3809862 by Steve.Robb

	GitHub #3342 : [FRotator.h] Fix to DecompressAxisFromByte to be more efficient and reflect its intent accurately

	#jira UE-42593

Change 3811190 by Graeme.Thornton

	Add support for writing specific log channels to their own files

Change 3811197 by Graeme.Thornton

	Minor updates to output formatting and timing for the text asset commandlet

Change 3811257 by Robert.Manuszewski

	Cluster creation will now be time-sliced

Change 3811565 by Steve.Robb

	Define out non-monolithic module functions.

Change 3812561 by Steve.Robb

	GitHub #3886 : Enable Brace-Initialization for Declaring Variables

	Incorrect semi-colon search removed after discussion with author.
	Test added.

	#jira UE-48242

Change 3812864 by Steve.Robb

	Removal of some unproven code which was supposed to fix hot reloading BP class functions in plugins.

	See: https://udn.unrealengine.com/questions/376978/aitask-blueprint-nodes-disappear-when-their-module.html

	#jira UE-53089

Change 3820358 by Ben.Marsh

	PR #4358: Incredibuild use ShowAgent by default (Contributed by projectgheist)


Change 3822594 by Ben.Marsh

	UAT: Improvements to log file handling.

	- Always create log files in the final location, rather than writing to a temp directory and copying in later.
	- Now supports -Verbose and -VeryVerbose for increasing log verbosity, rather than -Verbose=XXX.
	- Keep a backlog of log output before the log system is initialized, and flush it to the log file once it is.
	- Allow buildmachines to specify the uebp_FinalLogFolder environment variable, which is used to form paths for display. When build machines copy log files elsewhere after UAT finishes (eg. a network share), this allows error messages to display the right location.

Change 3823695 by Ben.Marsh

	UGS: Fix issue where precompiled binaries would not be shown as available for a change until scrolling the last submitted code change into the buffer (other symptoms, like de-focussing the main window would cause it to go back to an unavailable state, since the changes buffer was shrunk).

	Now always queries changes up to the last change for which zipped binaries are available.

Change 3823845 by Ben.Marsh

	UBT: Exclude C# projects for unsupported platforms when generating project files.

Change 3824180 by Ben.Marsh

	UGS: Add an option to show changes by build machines, and move the "only show reviewed" option in there too (Options > Show Changes).

	#jira

Change 3825777 by Steve.Robb

	Fix to return value of StringToBytes.

Change 3825810 by Ben.Marsh

	UBT: Reduce length of include paths for MSVC toolchain.

Change 3825822 by Robert.Manuszewski

	Optimized PIE lazy pointer fixup. Should be up to 8x faster now.

Change 3826734 by Ben.Marsh

	Remove code to disable TextureFormatAndroid on Linux. It seems to be an editor dependency.

Change 3827730 by Steve.Robb

	Try to avoid decltype(auto) if it's not supported.

	See: https://udn.unrealengine.com/questions/395644/build-417-with-c11-on-linux-ttuple-errors.html

Change 3827745 by Steve.Robb

	Initializer list support for TMap.

Change 3827770 by Steve.Robb

	GitHub #4399 : Added a CONSTEXPR qualifiers to FVariant::GetType()

	#jira UE-53813

Change 3829189 by Ben.Marsh

	UBT: Now always writes a minimal log file. By default, just contains the regular console output and any reasons why actions are outdated and needed to be executed. UAT directs child UBT instances to output logs into its own log folder, so that build machines can save them off.

Change 3830444 by Steve.Robb

	BuildVersion and ModuleManifest moved to Core, and parsing of these files reimplemented to avoid a JSON library.
	This should be revisited when Core has its own JSON library.

Change 3830718 by Ben.Marsh

	Fix incorrect group name being returned by FStatNameAndInfo::GetGroupName() for stat groups.

	The editor populates the viewport stats list by calling this for every registered stat and stat group (via FLevelViewportCommands::HandleNewStatGroup). The menu entry attempts to show the stat name with STAT_XXX stripped from the start as the menu item label, with the free-form text description as a tooltip.

	For stat groups, the it would previously just return the stat group name as "Groups" (due to the raw naming convention of "//Groups//STATGROUP_Foo//..."). Since this didn't match the expected naming convention in FLevelViewportCommands::HandleNewStat (ie. STAT_XXX or STATGROUP_XXX), it would fail to add it.

	When the first actual stat belonging to that group is added, it would add a menu entry for the group based on that, but the stat description no longer makes sense as a tooltip for the group. As a result, all the editor tooltips were junk.

	#jira UE-53845

Change 3831064 by Ben.Marsh

	Fix log file contention when spawning UBT recursively.

Change 3832654 by Ben.Marsh

	UGS: Fix error panel not being selected when opened, and weird alignment/color issues on it.

Change 3832680 by Ben.Marsh

	UGS: Fix failing to detect workspace if synced to a different stream. Seems to be a regression caused by recent P4D upgrade.

Change 3832695 by Ben.Marsh

	UGS: Invert the options in the 'Show Changes' submenu for simplicity.

Change 3833528 by Ben.Marsh

	UAT: Script to rewrite source files with public include paths relative to the 'Public' folder. Usage is: RebasePublicIncludePaths -UpdateDir=<Dir> [-Project=<Dir>] [-Write].

Change 3833543 by Ben.Marsh

	UBT: Allow targets to opt-out of having public include paths added for every dependent module. This reduces the command line length when building a target, which has recently become a problem with larger games (due to Microsoft's compiler embedding the command line into each object file, with a maximum length of 64kb). All engine modules are compiled with this enabled; games may opt into it by setting bLegacyPublicIncludePaths = false; from their .target.cs, as may individual modules.

Change 3834354 by Robert.Manuszewski

	Archetype pointer will now be cached to avoid locking the object tables when acquiring its info. It should also be faster this way regardless of any locks.

	#jira UE-52035

Change 3834400 by Robert.Manuszewski

	Fixing crash on exit caused by cached archetypes not being cleaned up before static exit cleanup.

	#jira UE-52035

Change 3834947 by Steve.Robb

	USE_FORMAT_STRING_TYPE_CHECKING removed from FMsg::Logf and FMsg::Logf_Internal.

Change 3835004 by Ben.Marsh

	Fix code that relies on dubious behavior of requiring referenced "include path only" modules having their _API macros set to be empty, even if the module is actually implemented in a separate DLL.

Change 3835340 by Ben.Marsh

	Fix errors making installed build from directories with spaces in the name.

Change 3835972 by Ben.Marsh

	UBT: Improved diagnostic message for targets which don't need a version file.

Change 3836019 by Ben.Marsh

	UBT: Fix warnings caused by defining linkage macros for third party libraries.

Change 3836269 by Ben.Marsh

	Fix message box larger than the screen height being created when a large number of modules are incompatible on startup.

Change 3836543 by Ben.Marsh

	Enable SoundMod plugin on Linux, since it's already supported through the editor.

Change 3836546 by Ben.Marsh

	PR #4412: fix type mismatch (Contributed by nakapon)


Change 3836805 by Ben.Marsh

	Fix commandlet to compile marketplace plugins.

Change 3836829 by Ben.Marsh

	UBT: Fix ability to precompile plugins from installed engine builds.

Change 3837036 by Ben.Marsh

	UBT: Write the previous and new contents of intermediate files to the log if they change. Makes it easier to debug unexpected rebuilds.

Change 3837037 by Ben.Marsh

	UBT: Fix engine modules having inconsistent definitions depending on whether modules are only referenced for their include paths vs being linked into a binary (due to different _API macro).

Change 3837040 by Ben.Marsh

	UBT: Remove code that initializes members in ModuleRules and TargetRules objects before the constructor is run. This is no longer necessary, now that the backwards-compatible default constructors have been removed.

Change 3837247 by Ben.Marsh

	UBT: Remove UELinkerFixups module, now that plugins and precompiled modules do not require hacks to force initialization (since they're linked in as object files).

	Encryption and signing keys are now set via macros expanded from the IMPLEMENT_PRIMARY_GAME_MODULE macro, via project-specific macros added in the TargetRules constructor.

Change 3837262 by Ben.Marsh

	UBT: Set whether a module is an engine module or not via a default value for the rules assembly. All non-program engine and enterprise modules are created with this flag set to true; program targets and modules are now created from a different assembly that sets it to false. This removes hacks from UEBuildModule needed to adjust behavior for different module types based on the directory containing the module.

	Also add a bUseBackwardsCompatibleDefaults flag to the TargetRules class, also initialized to a default value from a setting passed to the RulesAssembly constructor. This controls whether modules created for the target should be configured to allow breaking changes to default settings, and is set to false for all engine targets, and true for all project targets.

Change 3837343 by Ben.Marsh

	UBT: Remove the OverrideExecutableFileExtension target property. Change the only current use for this (the MayaLiveLinkPlugin target) to use a post build step to copy the file instead.

Change 3837356 by Ben.Marsh

	Fix invalid character encodings.

Change 3837727 by Graeme.Thornton

	UnrealPak: KeyGenerator: Only generate prime table when required, not all the time

Change 3837823 by Ben.Marsh

	UBT: Output warnings and errors when compiling module rules assembly in a way that allows them to be double-clicked in the Visual Studio output window.

Change 3837831 by Graeme.Thornton

	UBT: When parsing crypto settings, always load legacy data first, then allow the new system to override it. Provides the same key backwards compatibility that the editor settings class gives

Change 3837857 by Robert.Manuszewski

	PR #4404: Make FGCArrayPool singleton global instead of per-CU (Contributed by mhutch)


Change 3837943 by Robert.Manuszewski

	PR #4405: Fix FGarbageCollectionTracer (Contributed by mhutch)


Change 3838451 by Ben.Marsh

	UBT: Fix exceptions thrown on a background thread while caching C++ includes not being caught and logged correctly. Now captures exceptions and re-throws on the main thread.

	#jira UE-53996

Change 3839519 by Ben.Marsh

	UBT: Simplify configuring bPrecompile and bUsePrecompile settings for modules. Each rules assembly can now be configured as installed, which defaults the module rules it creates to use precompiled data.

Change 3843790 by Graeme.Thornton

	UnrealPak: Log the size of all encrypted data

Change 3844258 by Ben.Marsh

	Fix plugin compile failure when created via new plugin wizard. Passing -plugin on the command line is unnecessary, and is now reserved for packaging external plugins for the marketplace.

	Also extend the length of time that the error toast stays visible, and don't delete the plugin on failure.

	#jira UE-54157

Change 3845796 by Ben.Marsh

	Workaround for slow performance of String.EndsWith() on Mono.

Change 3845823 by Ben.Marsh

	Fix case sensitive matching of platform names in -TargetPlatform=X argument to BuildCookRun.

	#jira UE-54123

Change 3845901 by Arciel.Rekman

	Linux: fix crash due to lambda lifetime issues (UE-54040).

	- The lambda goes out of scope in FBufferVisualizationMenuCommands::CreateVisualizationCommands, crashing the editor if compiled with a recent clang (5.0+).

	(Edigrating 3819174 to Dev-Core)

Change 3846439 by Ben.Marsh

	Revert CL 3822742 to always call Process.WaitForExit(). The Android target platform module in the editor spawns ADB.EXE, which inherits the editor's stdout/stderr handles and forks itself. Process.WaitForExit() waits for EOF on those pipes, which never occurs because the forked process never terminates.

	Proper fix is probably to have the engine explicitly duplicate stdout/stderr handles for new pipes to output process, but too risky before copying up to Main.

Change 3816608 by Ben.Marsh

	UBT: Use DirectoryReference objects for all include paths.

Change 3816954 by Ben.Marsh

	UBT: Remove bIncludeDependentLibrariesInLibrary option. This is not widely supported by platform toolchains, and is not used anywhere.

Change 3816986 by Ben.Marsh

	UBT: Remove UEBuildBinaryConfig; UEBuildBinary objects are now just created directly.

Change 3816991 by Ben.Marsh

	UBT: Deprecate PlatformSpecificDynamicallyLoadedModules. We no longer have any special behavior for these modules.

Change 3823090 by Ben.Marsh

	UAT: Improve logging for child UAT instances.

	- Calling RunUAT now requires an identifier for prefixing into the parent log, which is also used to determine the name of the log folder.
	- Stdout is no longer written to its own output file, since it's written to the parent stdout, the parent log file, and the child log file anyway.
	- Log folders for child UAT instances are left intact, rather than being copied to the parent folder. The derived names for the copied names were confusing and hard to read.
	- Output from UAT is no longer returned as a string. It should not be parsed anyway (but may be huge!). ProcessResult now supports running without capturing output.

Change 3826082 by Ben.Marsh

	UBT: Add a check to make sure that all modules that are precompiled are correctly marked to enable it, even if they are part of the build target.

Change 3827025 by Ben.Marsh

	UBT: Move the compile output directory into a property on the module, and explicitly pass it to the toolchain when compiling.

Change 3829927 by James.Hopkin

	Made HTTP interface const correct

Change 3833533 by Ben.Marsh

	Rewrite engine source files to base include paths relative to the "Public" directory. This allows reducing the number of public include paths that have to be added for engine modules.

Change 3835826 by Ben.Marsh

	UBT: Precompiled targets now generate a separate manifest for each precompiled module, rather than adding object files to a library. This fixes issues where object files from static libraries would not be linked into a target if a symbol in them was not referenced.

Change 3835969 by Ben.Marsh

	UBT: Fix cases where text is being written directly to the console rather than via logging functions.

Change 3837777 by Steve.Robb

	Format string type checking added to FOutputDevice::Logf.
	Fixes for those.

Change 3838569 by Steve.Robb

	Algo moved up a folder.

[CL 3847482 by Ben Marsh in Main branch]
2018-01-20 11:19:29 -05:00

2091 lines
69 KiB
C++

// Copyright 1998-2018 Epic Games, Inc. All Rights Reserved.
#include "CoreMinimal.h"
#include "HAL/PlatformProcess.h"
#include "GenericPlatform/GenericPlatformFile.h"
#include "HAL/FileManager.h"
#include "Misc/CoreMisc.h"
#include "Misc/Paths.h"
#include "Misc/QueuedThreadPool.h"
#include "Misc/OutputDeviceNull.h"
#include "Stats/Stats.h"
#include "Async/AsyncWork.h"
#include "Containers/Ticker.h"
#include "Misc/ConfigCacheIni.h"
#include "Misc/FeedbackContext.h"
#include "Misc/ScopedSlowTask.h"
#include "Misc/App.h"
#include "Modules/ModuleManager.h"
#include "UObject/ObjectMacros.h"
#include "UObject/UObjectGlobals.h"
#include "Serialization/ArchiveUObject.h"
#include "UObject/GarbageCollection.h"
#include "UObject/Class.h"
#include "UObject/UObjectIterator.h"
#include "UObject/UnrealType.h"
#include "Misc/PackageName.h"
#include "IHotReload.h"
#include "IDirectoryWatcher.h"
#include "DirectoryWatcherModule.h"
#include "HotReloadLog.h"
#include "AnalyticsEventAttribute.h"
#include "Interfaces/IAnalyticsProvider.h"
#include "ProfilingDebugging/ScopedTimers.h"
#include "Interfaces/IPluginManager.h"
#include "DesktopPlatformModule.h"
#if WITH_ENGINE
#include "Engine/Engine.h"
#include "Kismet2/KismetReinstanceUtilities.h"
#include "HotReloadClassReinstancer.h"
#include "EngineAnalytics.h"
#endif
#include "Misc/ScopeExit.h"
#if WITH_EDITOR
#include "Editor.h"
#endif
DEFINE_LOG_CATEGORY(LogHotReload);
#define LOCTEXT_NAMESPACE "HotReload"
namespace EThreeStateBool
{
enum Type
{
False,
True,
Unknown
};
static bool ToBool(EThreeStateBool::Type Value)
{
switch (Value)
{
case EThreeStateBool::False:
return false;
case EThreeStateBool::True:
return true;
default:
UE_LOG(LogHotReload, Fatal, TEXT("Can't convert EThreeStateBool to bool value because it's Unknown"));
break;
}
return false;
}
static EThreeStateBool::Type FromBool(bool Value)
{
return Value ? EThreeStateBool::True : EThreeStateBool::False;
}
};
/**
* Module for HotReload support
*/
class FHotReloadModule : public IHotReloadModule, FSelfRegisteringExec
{
public:
FHotReloadModule()
{
ModuleCompileReadPipe = nullptr;
bRequestCancelCompilation = false;
bIsAnyGameModuleLoaded = EThreeStateBool::Unknown;
bDirectoryWatcherInitialized = false;
}
/** IModuleInterface implementation */
virtual void StartupModule() override;
virtual void ShutdownModule() override;
/** FSelfRegisteringExec implementation */
virtual bool Exec( UWorld* Inworld, const TCHAR* Cmd, FOutputDevice& Ar ) override;
/** IHotReloadInterface implementation */
virtual void SaveConfig() override;
virtual bool RecompileModule(const FName InModuleName, const bool bReloadAfterRecompile, FOutputDevice &Ar, bool bFailIfGeneratedCodeChanges = true, bool bForceCodeProject = false) override;
virtual bool IsCurrentlyCompiling() const override { return ModuleCompileProcessHandle.IsValid(); }
virtual void RequestStopCompilation() override { bRequestCancelCompilation = true; }
virtual void AddHotReloadFunctionRemap(FNativeFuncPtr NewFunctionPointer, FNativeFuncPtr OldFunctionPointer) override;
virtual ECompilationResult::Type RebindPackages(const TArray<UPackage*>& Packages, EHotReloadFlags Flags, FOutputDevice &Ar) override;
virtual ECompilationResult::Type DoHotReloadFromEditor(EHotReloadFlags Flags) override;
virtual FHotReloadEvent& OnHotReload() override { return HotReloadEvent; }
virtual FModuleCompilerStartedEvent& OnModuleCompilerStarted() override { return ModuleCompilerStartedEvent; }
virtual FModuleCompilerFinishedEvent& OnModuleCompilerFinished() override { return ModuleCompilerFinishedEvent; }
virtual FString GetModuleCompileMethod(FName InModuleName) override;
virtual bool IsAnyGameModuleLoaded() override;
private:
/**
* Enumerates compilation methods for modules.
*/
enum class EModuleCompileMethod
{
Runtime,
External,
Unknown
};
/**
* Helper structure to hold on to module state while asynchronously recompiling DLLs
*/
struct FModuleToRecompile
{
/** Name of the module */
FString ModuleName;
/** Desired module file name suffix, or empty string if not needed */
FString ModuleFileSuffix;
/** The module file name to use after a compilation succeeds, or an empty string if not changing */
FString NewModuleFilename;
};
/**
* Helper structure to store the compile time and method for a module
*/
struct FModuleCompilationData
{
/** Has a timestamp been set for the .dll file */
bool bHasFileTimeStamp;
/** Last known timestamp for the .dll file */
FDateTime FileTimeStamp;
/** Last known compilation method of the .dll file */
EModuleCompileMethod CompileMethod;
FModuleCompilationData()
: bHasFileTimeStamp(false)
, CompileMethod(EModuleCompileMethod::Unknown)
{ }
};
/**
* Adds a callback to directory watcher for the game binaries folder.
*/
void RefreshHotReloadWatcher();
/**
* Adds a directory watch on the binaries directory under the given folder.
*/
void AddHotReloadDirectory(IDirectoryWatcher* DirectoryWatcher, const FString& BaseDir);
/**
* Removes a directory watcher callback
*/
void ShutdownHotReloadWatcher();
/**
* Performs hot-reload from IDE (when game DLLs change)
*/
void DoHotReloadFromIDE(const TMap<FString, FString>& NewModules);
/**
* Performs internal module recompilation
*/
ECompilationResult::Type RebindPackagesInternal(const TArray<UPackage*>& Packages, const TArray<FName>& DependentModules, EHotReloadFlags Flags, FOutputDevice& Ar);
/**
* Does the actual hot-reload, unloads old modules, loads new ones
*/
ECompilationResult::Type DoHotReloadInternal(const TMap<FString, FString>& ChangedModuleNames, const TArray<UPackage*>& Packages, const TArray<FName>& InDependentModules, FOutputDevice& HotReloadAr);
/**
* Finds all references to old CDOs and replaces them with the new ones.
* Skipping UBlueprintGeneratedClass::OverridenArchetypeForCDO as it's the
* only one needed.
*/
void ReplaceReferencesToReconstructedCDOs();
#if WITH_ENGINE
void RegisterForReinstancing(UClass* OldClass, UClass* NewClass);
void ReinstanceClasses();
/**
* Called from CoreUObject to re-instance hot-reloaded classes
*/
void ReinstanceClass(UClass* OldClass, UClass* NewClass, const TMap<UClass*, UClass*>& OldToNewClassesMap);
#endif
/**
* Tick function for FTicker: checks for re-loaded modules and does hot-reload from IDE
*/
bool Tick(float DeltaTime);
/**
* Directory watcher callback
*/
void OnHotReloadBinariesChanged(const TArray<struct FFileChangeData>& FileChanges);
/**
* Strips hot-reload suffix from module filename.
*/
static void StripModuleSuffixFromFilename(FString& InOutModuleFilename, const FString& ModuleName);
/**
* Sends analytics event about the re-load
*/
static void RecordAnalyticsEvent(const TCHAR* ReloadFrom, ECompilationResult::Type Result, double Duration, int32 PackageCount, int32 DependentModulesCount);
/**
* Declares a function type that is executed after a module recompile has finished.
*
* ChangedModules: A map between the names of the modules that have changed and their filenames.
* bRecompileFinished: Signals whether compilation has finished.
* CompilationResult: Shows whether compilation was successful or not.
*/
typedef TFunction<void(const TMap<FString, FString>& ChangedModules, bool bRecompileFinished, ECompilationResult::Type CompilationResult)> FRecompileModulesCallback;
/** Called for successfully re-complied module */
void OnModuleCompileSucceeded(FName ModuleName, const FString& ModuleFilename);
/** Returns arguments to pass to UnrealBuildTool when compiling modules */
static FString MakeUBTArgumentsForModuleCompiling();
#if WITH_HOT_RELOAD
/**
* Starts compiling DLL files for one or more modules.
*
* @param GameName The name of the game.
* @param ModuleNames The list of modules to compile.
* @param InRecompileModulesCallback Callback function to make when module recompiles.
* @param Ar
* @param bInFailIfGeneratedCodeChanges If true, fail the compilation if generated headers change.
* @param InAdditionalCmdLineArgs Additional arguments to pass to UBT.
* @param bForceCodeProject Compile as code-based project even if there's no game modules loaded
* @return true if successful, false otherwise.
*/
bool StartCompilingModuleDLLs(const FString& GameName, const TArray< FModuleToRecompile >& ModuleNames,
FRecompileModulesCallback&& InRecompileModulesCallback, FOutputDevice& Ar, bool bInFailIfGeneratedCodeChanges,
const FString& InAdditionalCmdLineArgs, bool bForceCodeProject);
#endif
/** Launches UnrealBuildTool with the specified command line parameters */
bool InvokeUnrealBuildToolForCompile(const FString& InCmdLineParams, FOutputDevice &Ar);
/** Checks to see if a pending compilation action has completed and optionally waits for it to finish. If completed, fires any appropriate callbacks and reports status provided bFireEvents is true. */
void CheckForFinishedModuleDLLCompile(EHotReloadFlags Flags, bool& bCompileStillInProgress, bool& bCompileSucceeded, FOutputDevice& Ar, bool bFireEvents = true);
/** Called when the compile data for a module need to be update in memory and written to config */
void UpdateModuleCompileData(FName ModuleName);
/** Called when a new module is added to the manager to get the saved compile data from config */
static void ReadModuleCompilationInfoFromConfig(FName ModuleName, FModuleCompilationData& CompileData);
/** Saves the module's compile data to config */
static void WriteModuleCompilationInfoToConfig(FName ModuleName, const FModuleCompilationData& CompileData);
/** Access the module's file and read the timestamp from the file system. Returns true if the timestamp was read successfully. */
bool GetModuleFileTimeStamp(FName ModuleName, FDateTime& OutFileTimeStamp) const;
/** Checks if the specified array of modules to recompile contains only game modules */
bool ContainsOnlyGameModules(const TArray< FModuleToRecompile >& ModuleNames) const;
/** Callback registered with ModuleManager to know if any new modules have been loaded */
void ModulesChangedCallback(FName ModuleName, EModuleChangeReason ReasonForChange);
/** Callback registered with PluginManager to know if any new plugins have been created */
void PluginMountedCallback(IPlugin& Plugin);
/** FTicker delegate (hot-reload from IDE) */
FTickerDelegate TickerDelegate;
/** Handle to the registered TickerDelegate */
FDelegateHandle TickerDelegateHandle;
/** Handle to the registered delegate above */
TMap<FString, FDelegateHandle> BinariesFolderChangedDelegateHandles;
/** True if currently hot-reloading from editor (suppresses hot-reload from IDE) */
bool bIsHotReloadingFromEditor;
/** New module DLLs detected by the directory watcher */
TMap<FString, FString> DetectedNewModules;
/** Moduels that have been recently recompiled from the editor **/
TSet<FString> ModulesRecentlyCompiledInTheEditor;
/** Delegate broadcast when a module has been hot-reloaded */
FHotReloadEvent HotReloadEvent;
/** Array of modules that we're currently recompiling */
TArray< FModuleToRecompile > ModulesBeingCompiled;
/** Array of modules that we're going to recompile */
TArray< FModuleToRecompile > ModulesThatWereBeingRecompiled;
/** Last known compilation data for each module */
TMap<FName, TSharedRef<FModuleCompilationData>> ModuleCompileData;
/** Multicast delegate which will broadcast a notification when the compiler starts */
FModuleCompilerStartedEvent ModuleCompilerStartedEvent;
/** Multicast delegate which will broadcast a notification when the compiler finishes */
FModuleCompilerFinishedEvent ModuleCompilerFinishedEvent;
/** When compiling a module using an external application, stores the handle to the process that is running */
FProcHandle ModuleCompileProcessHandle;
/** When compiling a module using an external application, this is the process read pipe handle */
void* ModuleCompileReadPipe;
/** When compiling a module using an external application, this is the text that was read from the read pipe handle */
FString ModuleCompileReadPipeText;
/** Callback to execute after an asynchronous recompile has completed (whether successful or not.) */
FRecompileModulesCallback RecompileModulesCallback;
/** true if we should attempt to cancel the current async compilation */
bool bRequestCancelCompilation;
/** Tracks the validity of the game module existence */
EThreeStateBool::Type bIsAnyGameModuleLoaded;
/** True if the directory watcher has been successfully initialized */
bool bDirectoryWatcherInitialized;
/** Reconstructed CDOs map during hot-reload. */
TMap<UObject*, UObject*> ReconstructedCDOsMap;
/** Keeps record of hot-reload session starting time. */
double HotReloadStartTime;
};
namespace HotReloadDefs
{
const static FString CompilationInfoConfigSection("ModuleFileTracking");
// These strings should match the values of the enum EModuleCompileMethod in ModuleManager.h
// and should be handled in ReadModuleCompilationInfoFromConfig() & WriteModuleCompilationInfoToConfig() below
const static FString CompileMethodRuntime("Runtime");
const static FString CompileMethodExternal("External");
const static FString CompileMethodUnknown("Unknown");
// Add one minute epsilon to timestamp comparision
const static FTimespan TimeStampEpsilon(0, 1, 0);
}
IMPLEMENT_MODULE(FHotReloadModule, HotReload);
namespace UE4HotReload_Private
{
/**
* Gets editor runs directory.
*/
FString GetEditorRunsDir()
{
FString TempDir = FPaths::EngineIntermediateDir();
return FPaths::Combine(*TempDir, TEXT("EditorRuns"));
}
/**
* Creates a file that informs UBT that the editor is currently running.
*/
void CreateFileThatIndicatesEditorRunIfNeeded()
{
#if WITH_EDITOR
IPlatformFile& FS = IPlatformFile::GetPlatformPhysical();
FString EditorRunsDir = GetEditorRunsDir();
FString FileName = FPaths::Combine(*EditorRunsDir, *FString::Printf(TEXT("%d"), FPlatformProcess::GetCurrentProcessId()));
if (FS.FileExists(*FileName))
{
if (!GIsEditor)
{
FS.DeleteFile(*FileName);
}
}
else
{
if (GIsEditor)
{
if (!FS.CreateDirectory(*EditorRunsDir))
{
return;
}
delete FS.OpenWrite(*FileName); // Touch file.
}
}
#endif // WITH_EDITOR
}
/**
* Deletes file left by CreateFileThatIndicatesEditorRunIfNeeded function.
*/
void DeleteFileThatIndicatesEditorRunIfNeeded()
{
#if WITH_EDITOR
IPlatformFile& FS = IPlatformFile::GetPlatformPhysical();
FString EditorRunsDir = GetEditorRunsDir();
FString FileName = FPaths::Combine(*EditorRunsDir, *FString::Printf(TEXT("%d"), FPlatformProcess::GetCurrentProcessId()));
if (FS.FileExists(*FileName))
{
FS.DeleteFile(*FileName);
}
#endif // WITH_EDITOR
}
/**
* Gets all currently loaded game module names and optionally, the file names for those modules
*/
TArray<FString> GetGameModuleNames(const FModuleManager& ModuleManager)
{
TArray<FString> Result;
// Ask the module manager for a list of currently-loaded gameplay modules
TArray<FModuleStatus> ModuleStatuses;
ModuleManager.QueryModules(ModuleStatuses);
for (FModuleStatus& ModuleStatus : ModuleStatuses)
{
// We only care about game modules that are currently loaded
if (ModuleStatus.bIsLoaded && ModuleStatus.bIsGameModule)
{
Result.Add(MoveTemp(ModuleStatus.Name));
}
}
return Result;
}
/**
* Gets all currently loaded game module names and optionally, the file names for those modules
*/
TMap<FString, FString> GetGameModuleFilenames(const FModuleManager& ModuleManager)
{
TMap<FString, FString> Result;
// Ask the module manager for a list of currently-loaded gameplay modules
TArray< FModuleStatus > ModuleStatuses;
ModuleManager.QueryModules(ModuleStatuses);
for (FModuleStatus& ModuleStatus : ModuleStatuses)
{
// We only care about game modules that are currently loaded
if (ModuleStatus.bIsLoaded && ModuleStatus.bIsGameModule)
{
Result.Add(MoveTemp(ModuleStatus.Name), MoveTemp(ModuleStatus.FilePath));
}
}
return Result;
}
struct FPackagesAndDependentNames
{
TArray<UPackage*> Packages;
TArray<FName> DependentNames;
};
/**
* Gets named packages and the names dependents.
*/
FPackagesAndDependentNames SplitByPackagesAndDependentNames(const TArray<FString>& ModuleNames)
{
FPackagesAndDependentNames Result;
for (const FString& ModuleName : ModuleNames)
{
FString PackagePath = TEXT("/Script/") + ModuleName;
if (UPackage* Package = FindPackage(nullptr, *PackagePath))
{
Result.Packages.Add(Package);
}
else
{
Result.DependentNames.Add(*ModuleName);
}
}
return Result;
}
}
void FHotReloadModule::StartupModule()
{
UE4HotReload_Private::CreateFileThatIndicatesEditorRunIfNeeded();
bIsHotReloadingFromEditor = false;
#if WITH_ENGINE
// Register re-instancing delegate (Core)
FCoreUObjectDelegates::RegisterClassForHotReloadReinstancingDelegate.AddRaw(this, &FHotReloadModule::RegisterForReinstancing);
FCoreUObjectDelegates::ReinstanceHotReloadedClassesDelegate.AddRaw(this, &FHotReloadModule::ReinstanceClasses);
#endif
// Register directory watcher delegate
RefreshHotReloadWatcher();
// Register hot-reload from IDE ticker
TickerDelegate = FTickerDelegate::CreateRaw(this, &FHotReloadModule::Tick);
TickerDelegateHandle = FTicker::GetCoreTicker().AddTicker(TickerDelegate);
FModuleManager::Get().OnModulesChanged().AddRaw(this, &FHotReloadModule::ModulesChangedCallback);
IPluginManager::Get().OnNewPluginMounted().AddRaw(this, &FHotReloadModule::PluginMountedCallback);
}
void FHotReloadModule::ShutdownModule()
{
FTicker::GetCoreTicker().RemoveTicker(TickerDelegateHandle);
ShutdownHotReloadWatcher();
UE4HotReload_Private::DeleteFileThatIndicatesEditorRunIfNeeded();
}
bool FHotReloadModule::Exec( UWorld* Inworld, const TCHAR* Cmd, FOutputDevice& Ar )
{
#if !UE_BUILD_SHIPPING
if ( FParse::Command( &Cmd, TEXT( "Module" ) ) )
{
#if WITH_HOT_RELOAD
// Recompile <ModuleName>
if( FParse::Command( &Cmd, TEXT( "Recompile" ) ) )
{
const FString ModuleNameStr = FParse::Token( Cmd, 0 );
if( !ModuleNameStr.IsEmpty() )
{
const FName ModuleName( *ModuleNameStr );
const bool bReloadAfterRecompile = true;
const bool bForceCodeProject = false;
const bool bFailIfGeneratedCodeChanges = true;
RecompileModule( ModuleName, bReloadAfterRecompile, Ar, bFailIfGeneratedCodeChanges, bForceCodeProject);
}
return true;
}
#endif // WITH_HOT_RELOAD
}
#endif // !UE_BUILD_SHIPPING
return false;
}
void FHotReloadModule::SaveConfig()
{
// Find all the modules
TArray<FModuleStatus> Modules;
FModuleManager::Get().QueryModules(Modules);
// Update the compile data for each one
for( const FModuleStatus &Module : Modules )
{
UpdateModuleCompileData(*Module.Name);
}
}
FString FHotReloadModule::GetModuleCompileMethod(FName InModuleName)
{
if (!ModuleCompileData.Contains(InModuleName))
{
UpdateModuleCompileData(InModuleName);
}
switch(ModuleCompileData.FindChecked(InModuleName).Get().CompileMethod)
{
case EModuleCompileMethod::External:
return HotReloadDefs::CompileMethodExternal;
case EModuleCompileMethod::Runtime:
return HotReloadDefs::CompileMethodRuntime;
default:
return HotReloadDefs::CompileMethodUnknown;
}
}
bool FHotReloadModule::RecompileModule(const FName InModuleName, const bool bReloadAfterRecompile, FOutputDevice &Ar, bool bFailIfGeneratedCodeChanges, bool bForceCodeProject)
{
#if WITH_HOT_RELOAD
UE_LOG(LogHotReload, Log, TEXT("Recompiling module %s..."), *InModuleName.ToString());
// This is an internal request for hot-reload (not from IDE)
TGuardValue<bool> GuardHotReloadingFromEditorFlag(bIsHotReloadingFromEditor, true);
// A list of modules that have been recompiled in the editor is going to prevent false
// hot-reload from IDE events as this call is blocking any potential callbacks coming from the filesystem
// and bIsHotReloadingFromEditor may not be enough to prevent those from being treated as actual hot-reload from IDE modules
ModulesRecentlyCompiledInTheEditor.Empty();
FFormatNamedArguments Args;
Args.Add( TEXT("CodeModuleName"), FText::FromName( InModuleName ) );
const FText StatusUpdate = FText::Format( NSLOCTEXT("ModuleManager", "Recompile_SlowTaskName", "Compiling {CodeModuleName}..."), Args );
FScopedSlowTask SlowTask(2, StatusUpdate);
SlowTask.MakeDialog();
ModuleCompilerStartedEvent.Broadcast(false); // we never perform an async compile
FModuleManager& ModuleManager = FModuleManager::Get();
// Update our set of known modules, in case we don't already know about this module
ModuleManager.AddModule( InModuleName );
// Only use rolling module names if the module was already loaded into memory. This allows us to try compiling
// the module without actually having to unload it first.
const bool bWasModuleLoaded = ModuleManager.IsModuleLoaded( InModuleName );
const bool bUseRollingModuleNames = bWasModuleLoaded;
SlowTask.EnterProgressFrame();
/**
* Tries to recompile the specified DLL using UBT. Does not interact with modules. This is a low level routine.
*
* @param ModuleNames List of modules to recompile, including the module name and optional file suffix.
* @param Ar Output device for logging compilation status.
* @param bForceCodeProject Even if it's a non-code project, treat it as code-based project
*/
auto RecompileModuleDLLs = [this, &Ar, bFailIfGeneratedCodeChanges, bForceCodeProject](const TArray< FModuleToRecompile >& ModuleNames)
{
bool bCompileSucceeded = false;
const FString AdditionalArguments = MakeUBTArgumentsForModuleCompiling();
if (StartCompilingModuleDLLs(FApp::GetProjectName(), ModuleNames, nullptr, Ar, bFailIfGeneratedCodeChanges, AdditionalArguments, bForceCodeProject))
{
bool bCompileStillInProgress = false;
CheckForFinishedModuleDLLCompile( EHotReloadFlags::WaitForCompletion, bCompileStillInProgress, bCompileSucceeded, Ar );
}
return bCompileSucceeded;
};
if( bUseRollingModuleNames )
{
// First, try to compile the module. If the module is already loaded, we won't unload it quite yet. Instead
// make sure that it compiles successfully.
// Find a unique file name for the module
FModuleToRecompile ModuleToRecompile;
ModuleToRecompile.ModuleName = InModuleName.ToString();
ModuleManager.MakeUniqueModuleFilename( InModuleName, ModuleToRecompile.ModuleFileSuffix, ModuleToRecompile.NewModuleFilename );
TArray< FModuleToRecompile > ModulesToRecompile;
ModulesToRecompile.Add( MoveTemp(ModuleToRecompile) );
ModulesRecentlyCompiledInTheEditor.Add(FPaths::ConvertRelativePathToFull(ModuleToRecompile.NewModuleFilename));
if (!RecompileModuleDLLs(ModulesToRecompile))
{
return false;
}
}
SlowTask.EnterProgressFrame();
// Shutdown the module if it's already running
if( bWasModuleLoaded )
{
Ar.Logf( TEXT( "Unloading module before compile." ) );
ModuleManager.UnloadOrAbandonModuleWithCallback( InModuleName, Ar );
}
if( !bUseRollingModuleNames )
{
// Try to recompile the DLL
TArray< FModuleToRecompile > ModulesToRecompile;
FModuleToRecompile ModuleToRecompile;
ModuleToRecompile.ModuleName = InModuleName.ToString();
if (ModuleManager.IsModuleLoaded(InModuleName))
{
ModulesRecentlyCompiledInTheEditor.Add(FPaths::ConvertRelativePathToFull(ModuleManager.GetModuleFilename(InModuleName)));
}
else
{
ModuleToRecompile.NewModuleFilename = ModuleManager.GetGameBinariesDirectory() / ModuleManager.GetCleanModuleFilename(InModuleName, true);
ModulesRecentlyCompiledInTheEditor.Add(FPaths::ConvertRelativePathToFull(ModuleToRecompile.NewModuleFilename));
}
ModulesToRecompile.Add( ModuleToRecompile );
if (!RecompileModuleDLLs(ModulesToRecompile))
{
return false;
}
}
// Reload the module if it was loaded before we recompiled
if ((bWasModuleLoaded || bForceCodeProject) && bReloadAfterRecompile)
{
TGuardValue<bool> GuardIsHotReload(GIsHotReload, true);
Ar.Logf( TEXT( "Reloading module %s after successful compile." ), *InModuleName.ToString() );
if (!ModuleManager.LoadModuleWithCallback( InModuleName, Ar ))
{
return false;
}
CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS);
}
if (bForceCodeProject)
{
HotReloadEvent.Broadcast( false );
}
return true;
#else
return false;
#endif // WITH_HOT_RELOAD
}
/** Type hash for a UObject Function Pointer, maybe not a great choice, but it should be sufficient for the needs here. **/
inline uint32 GetTypeHash(FNativeFuncPtr A)
{
return *(uint32*)&A;
}
/** Map from old function pointer to new function pointer for hot reload. */
static TMap<FNativeFuncPtr, FNativeFuncPtr> HotReloadFunctionRemap;
static TSet<UBlueprint*> HotReloadBPSetToRecompile;
static TSet<UBlueprint*> HotReloadBPSetToRecompileBytecodeOnly;
/** Adds and entry for the UFunction native pointer remap table */
void FHotReloadModule::AddHotReloadFunctionRemap(FNativeFuncPtr NewFunctionPointer, FNativeFuncPtr OldFunctionPointer)
{
FNativeFuncPtr OtherNewFunction = HotReloadFunctionRemap.FindRef(OldFunctionPointer);
check(!OtherNewFunction || OtherNewFunction == NewFunctionPointer);
check(NewFunctionPointer);
check(OldFunctionPointer);
HotReloadFunctionRemap.Add(OldFunctionPointer, NewFunctionPointer);
}
ECompilationResult::Type FHotReloadModule::DoHotReloadFromEditor(EHotReloadFlags Flags)
{
// Get all game modules we want to compile
const FModuleManager& ModuleManager = FModuleManager::Get();
TArray<FString> GameModuleNames = UE4HotReload_Private::GetGameModuleNames(ModuleManager);
ECompilationResult::Type Result = ECompilationResult::Unsupported;
UE4HotReload_Private::FPackagesAndDependentNames PackagesAndDependentNames = UE4HotReload_Private::SplitByPackagesAndDependentNames(GameModuleNames);
// Analytics
double Duration = 0.0;
{
FScopedDurationTimer Timer(Duration);
Result = RebindPackagesInternal(PackagesAndDependentNames.Packages, PackagesAndDependentNames.DependentNames, Flags, *GLog);
}
RecordAnalyticsEvent(TEXT("Editor"), Result, Duration, PackagesAndDependentNames.Packages.Num(), PackagesAndDependentNames.DependentNames.Num());
return Result;
}
ECompilationResult::Type FHotReloadModule::DoHotReloadInternal(const TMap<FString, FString>& ChangedModules, const TArray<UPackage*>& Packages, const TArray<FName>& InDependentModules, FOutputDevice& HotReloadAr)
{
#if WITH_HOT_RELOAD
FModuleManager& ModuleManager = FModuleManager::Get();
ModuleManager.ResetModulePathsCache();
FFeedbackContext& ErrorsFC = UClass::GetDefaultPropertiesFeedbackContext();
ErrorsFC.ClearWarningsAndErrors();
// Rebind the hot reload DLL
TGuardValue<bool> GuardIsHotReload(GIsHotReload, true);
TGuardValue<bool> GuardIsInitialLoad(GIsInitialLoad, true);
HotReloadFunctionRemap.Empty(); // redundant
CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS); // we create a new CDO in the transient package...this needs to go away before we try again.
// Load the new modules up
bool bReloadSucceeded = false;
ECompilationResult::Type Result = ECompilationResult::Unsupported;
for (UPackage* Package : Packages)
{
FString PackageName = Package->GetName();
FString ShortPackageName = FPackageName::GetShortName(PackageName);
if (!ChangedModules.Contains(ShortPackageName))
{
continue;
}
FName ShortPackageFName = *ShortPackageName;
// Abandon the old module. We can't unload it because various data structures may be living
// that have vtables pointing to code that would become invalidated.
ModuleManager.AbandonModuleWithCallback(ShortPackageFName);
// Load the newly-recompiled module up (it will actually have a different DLL file name at this point.)
bReloadSucceeded = ModuleManager.LoadModule(ShortPackageFName) != nullptr;
if (!bReloadSucceeded)
{
HotReloadAr.Logf(ELogVerbosity::Warning, TEXT("HotReload failed, reload failed %s."), *PackageName);
Result = ECompilationResult::OtherCompilationError;
break;
}
}
// Load dependent modules.
for (FName ModuleName : InDependentModules)
{
FString ModuleNameStr = ModuleName.ToString();
if (!ChangedModules.Contains(ModuleNameStr))
{
continue;
}
ModuleManager.UnloadOrAbandonModuleWithCallback(ModuleName, HotReloadAr);
const bool bLoaded = ModuleManager.LoadModuleWithCallback(ModuleName, HotReloadAr);
if (!bLoaded)
{
HotReloadAr.Logf(ELogVerbosity::Warning, TEXT("Unable to reload module %s"), *ModuleName.GetPlainNameString());
}
}
if (ErrorsFC.GetNumErrors() || ErrorsFC.GetNumWarnings())
{
TArray<FString> AllErrorsAndWarnings;
ErrorsFC.GetErrorsAndWarningsAndEmpty(AllErrorsAndWarnings);
FString AllInOne;
for (const FString& ErrorOrWarning : AllErrorsAndWarnings)
{
AllInOne += ErrorOrWarning;
AllInOne += TEXT("\n");
}
HotReloadAr.Logf(ELogVerbosity::Warning, TEXT("Some classes could not be reloaded:\n%s"), *AllInOne);
}
if (bReloadSucceeded)
{
int32 NumFunctionsRemapped = 0;
// Remap all native functions (and gather scriptstructs)
TArray<UScriptStruct*> ScriptStructs;
for (FRawObjectIterator It; It; ++It)
{
if (UFunction* Function = Cast<UFunction>(static_cast<UObject*>(It->Object)))
{
if (FNativeFuncPtr NewFunction = HotReloadFunctionRemap.FindRef(Function->GetNativeFunc()))
{
++NumFunctionsRemapped;
Function->SetNativeFunc(NewFunction);
}
}
if (UScriptStruct* ScriptStruct = Cast<UScriptStruct>(static_cast<UObject*>(It->Object)))
{
if (!ScriptStruct->HasAnyFlags(RF_ClassDefaultObject) && ScriptStruct->GetCppStructOps() && Packages.ContainsByPredicate([=](UPackage* Package) { return ScriptStruct->IsIn(Package); }))
{
ScriptStructs.Add(ScriptStruct);
}
}
}
// now let's set up the script structs...this relies on super behavior, so null them all, then set them all up. Internally this sets them up hierarchically.
for (UScriptStruct* Script : ScriptStructs)
{
Script->ClearCppStructOps();
}
for (UScriptStruct* Script : ScriptStructs)
{
Script->PrepareCppStructOps();
check(Script->GetCppStructOps());
}
// Make sure new classes have the token stream assembled
UClass::AssembleReferenceTokenStreams();
HotReloadAr.Logf(ELogVerbosity::Display, TEXT("HotReload successful (%d functions remapped %d scriptstructs remapped)"), NumFunctionsRemapped, ScriptStructs.Num());
HotReloadFunctionRemap.Empty();
ReplaceReferencesToReconstructedCDOs();
// Force GC to collect reinstanced objects
CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS, true);
Result = ECompilationResult::Succeeded;
}
HotReloadEvent.Broadcast( !bIsHotReloadingFromEditor );
HotReloadAr.Logf(ELogVerbosity::Display, TEXT("HotReload took %4.1fs."), FPlatformTime::Seconds() - HotReloadStartTime);
bIsHotReloadingFromEditor = false;
return Result;
#else
bIsHotReloadingFromEditor = false;
return ECompilationResult::Unsupported;
#endif
}
void FHotReloadModule::ReplaceReferencesToReconstructedCDOs()
{
if (ReconstructedCDOsMap.Num() == 0)
{
return;
}
// Thread pool manager. We need new thread pool with increased
// amount of stack size. Standard GThreadPool was encountering
// stack overflow error during serialization.
static struct FReplaceReferencesThreadPool
{
FReplaceReferencesThreadPool()
{
Pool = FQueuedThreadPool::Allocate();
int32 NumThreadsInThreadPool = FPlatformMisc::NumberOfWorkerThreadsToSpawn();
verify(Pool->Create(NumThreadsInThreadPool, 256 * 1024));
}
~FReplaceReferencesThreadPool()
{
Pool->Destroy();
}
FQueuedThreadPool* GetPool() { return Pool; }
private:
FQueuedThreadPool* Pool;
} ThreadPoolManager;
// Async task to enable multithreaded CDOs reference search.
class FFindRefTask : public FNonAbandonableTask
{
public:
explicit FFindRefTask(const TMap<UObject*, UObject*>& InReconstructedCDOsMap, int32 ReserveElements)
: ReconstructedCDOsMap(InReconstructedCDOsMap)
{
ObjectsArray.Reserve(ReserveElements);
}
void DoWork()
{
for (UObject* Object : ObjectsArray)
{
class FReplaceCDOReferencesArchive : public FArchiveUObject
{
public:
FReplaceCDOReferencesArchive(UObject* InPotentialReferencer, const TMap<UObject*, UObject*>& InReconstructedCDOsMap)
: ReconstructedCDOsMap(InReconstructedCDOsMap)
, PotentialReferencer(InPotentialReferencer)
{
ArIsObjectReferenceCollector = true;
ArIgnoreOuterRef = true;
}
virtual FString GetArchiveName() const override
{
return TEXT("FReplaceCDOReferencesArchive");
}
FArchive& operator<<(UObject*& ObjRef)
{
UObject* Obj = ObjRef;
if (Obj && Obj != PotentialReferencer)
{
if (UObject* const* FoundObj = ReconstructedCDOsMap.Find(Obj))
{
ObjRef = *FoundObj;
}
}
return *this;
}
const TMap<UObject*, UObject*>& ReconstructedCDOsMap;
UObject* PotentialReferencer;
};
FReplaceCDOReferencesArchive FindRefsArchive(Object, ReconstructedCDOsMap);
Object->Serialize(FindRefsArchive);
}
}
FORCEINLINE TStatId GetStatId() const
{
RETURN_QUICK_DECLARE_CYCLE_STAT(FFindRefTask, STATGROUP_ThreadPoolAsyncTasks);
}
TArray<UObject*> ObjectsArray;
private:
const TMap<UObject*, UObject*>& ReconstructedCDOsMap;
};
const int32 NumberOfThreads = FPlatformMisc::NumberOfWorkerThreadsToSpawn();
const int32 NumObjects = GUObjectArray.GetObjectArrayNum();
const int32 ObjectsPerTask = FMath::CeilToInt((float)NumObjects / NumberOfThreads);
// Create tasks.
TArray<FAsyncTask<FFindRefTask>> Tasks;
Tasks.Reserve(NumberOfThreads);
for (int32 TaskId = 0; TaskId < NumberOfThreads; ++TaskId)
{
Tasks.Emplace(ReconstructedCDOsMap, ObjectsPerTask);
}
// Distribute objects uniformly between tasks.
int32 CurrentTaskId = 0;
for (FObjectIterator ObjIter; ObjIter; ++ObjIter)
{
UObject* CurObject = *ObjIter;
if (CurObject->IsPendingKill())
{
continue;
}
Tasks[CurrentTaskId].GetTask().ObjectsArray.Add(CurObject);
CurrentTaskId = (CurrentTaskId + 1) % NumberOfThreads;
}
// Run async tasks in worker threads.
for (FAsyncTask<FFindRefTask>& Task : Tasks)
{
Task.StartBackgroundTask(ThreadPoolManager.GetPool());
}
// Wait until tasks are finished
for (FAsyncTask<FFindRefTask>& AsyncTask : Tasks)
{
AsyncTask.EnsureCompletion();
}
ReconstructedCDOsMap.Empty();
}
ECompilationResult::Type FHotReloadModule::RebindPackages(const TArray<UPackage*>& InPackages, EHotReloadFlags Flags, FOutputDevice &Ar)
{
ECompilationResult::Type Result = ECompilationResult::Unknown;
// Get game packages
const FModuleManager& ModuleManager = FModuleManager::Get();
TArray<FString> GameModuleNames = UE4HotReload_Private::GetGameModuleNames(ModuleManager);
UE4HotReload_Private::FPackagesAndDependentNames PackagesAndDependentNames = UE4HotReload_Private::SplitByPackagesAndDependentNames(GameModuleNames);
// Get a set of source packages combined with game packages
TSet<UPackage*> PackagesIncludingGame(InPackages);
int32 NumInPackages = PackagesIncludingGame.Num();
PackagesIncludingGame.Append(PackagesAndDependentNames.Packages);
// Check if there was any overlap
bool bInPackagesIncludeGame = PackagesIncludingGame.Num() < NumInPackages + PackagesAndDependentNames.Packages.Num();
// If any of those modules were game modules, we'll compile those too
TArray<UPackage*> Packages;
TArray<FName> Dependencies;
if (bInPackagesIncludeGame)
{
Packages = PackagesIncludingGame.Array();
Dependencies = MoveTemp(PackagesAndDependentNames.DependentNames);
}
else
{
Packages = InPackages;
}
double Duration = 0.0;
{
FScopedDurationTimer RebindTimer(Duration);
Result = RebindPackagesInternal(Packages, Dependencies, Flags, Ar);
}
RecordAnalyticsEvent(TEXT("Rebind"), Result, Duration, Packages.Num(), Dependencies.Num());
return Result;
}
ECompilationResult::Type FHotReloadModule::RebindPackagesInternal(const TArray<UPackage*>& InPackages, const TArray<FName>& DependentModules, EHotReloadFlags Flags, FOutputDevice& Ar)
{
#if WITH_HOT_RELOAD
if (InPackages.Num() == 0)
{
Ar.Logf(ELogVerbosity::Warning, TEXT("RebindPackages not possible (no packages specified)"));
return ECompilationResult::Unsupported;
}
// Verify that we're going to be able to rebind the specified packages
for (UPackage* Package : InPackages)
{
check(Package);
if (Package->GetOuter())
{
Ar.Logf(ELogVerbosity::Warning, TEXT("Could not rebind package for %s, package is either not bound yet or is not a DLL."), *Package->GetName());
return ECompilationResult::Unsupported;
}
}
// We can only proceed if a compile isn't already in progress
if (IsCurrentlyCompiling())
{
Ar.Logf(ELogVerbosity::Warning, TEXT("Could not rebind package because a module compile is already in progress."));
return ECompilationResult::Unsupported;
}
FModuleManager::Get().ResetModulePathsCache();
bIsHotReloadingFromEditor = true;
HotReloadStartTime = FPlatformTime::Seconds();
TArray< FName > ModuleNames;
for (UPackage* Package : InPackages)
{
// Attempt to recompile this package's module
FName ShortPackageName = FPackageName::GetShortFName(Package->GetFName());
ModuleNames.Add(ShortPackageName);
}
// Add dependent modules.
ModuleNames.Append(DependentModules);
// Start compiling modules
//
// NOTE: This method of recompiling always using a rolling file name scheme, since we never want to unload before
// we start recompiling, and we need the output DLL to be unlocked before we invoke the compiler
ModuleCompilerStartedEvent.Broadcast(!(Flags & EHotReloadFlags::WaitForCompletion)); // we perform an async compile providing we're not waiting for completion
FModuleManager& ModuleManager = FModuleManager::Get();
TArray< FModuleToRecompile > ModulesToRecompile;
for( FName CurModuleName : ModuleNames )
{
// Update our set of known modules, in case we don't already know about this module
ModuleManager.AddModule( CurModuleName );
// Find a unique file name for the module
FModuleToRecompile ModuleToRecompile;
ModuleToRecompile.ModuleName = CurModuleName.ToString();
ModuleManager.MakeUniqueModuleFilename( CurModuleName, ModuleToRecompile.ModuleFileSuffix, ModuleToRecompile.NewModuleFilename );
ModulesToRecompile.Add( ModuleToRecompile );
}
// Kick off compilation!
const FString AdditionalArguments = MakeUBTArgumentsForModuleCompiling();
bool bCompileStarted = StartCompilingModuleDLLs(
FApp::GetProjectName(),
ModulesToRecompile,
[this, InPackages, DependentModules, &Ar](const TMap<FString, FString>& ChangedModules, bool bRecompileFinished, ECompilationResult::Type CompilationResult)
{
if (ECompilationResult::Failed(CompilationResult) && bRecompileFinished)
{
Ar.Logf(ELogVerbosity::Warning, TEXT("HotReload failed, recompile failed"));
return;
}
DoHotReloadInternal(ChangedModules, InPackages, DependentModules, Ar);
},
Ar,
false, /* bFailIfGeneratedCodeChanges */
AdditionalArguments,
false /* bForceCodeProject */
);
if (!bCompileStarted)
{
Ar.Logf(ELogVerbosity::Warning, TEXT("RebindPackages failed because the compiler could not be started."));
bIsHotReloadingFromEditor = false;
return ECompilationResult::OtherCompilationError;
}
// Go ahead and check for completion right away. This is really just so that we can handle the case
// where the user asked us to wait for the compile to finish before returning.
if (!!(Flags & EHotReloadFlags::WaitForCompletion))
{
bool bCompileStillInProgress = false;
bool bCompileSucceeded = false;
FOutputDeviceNull NullOutput;
CheckForFinishedModuleDLLCompile( Flags, bCompileStillInProgress, bCompileSucceeded, NullOutput );
if( !bCompileStillInProgress && !bCompileSucceeded )
{
Ar.Logf(ELogVerbosity::Warning, TEXT("RebindPackages failed because compilation failed."));
bIsHotReloadingFromEditor = false;
return ECompilationResult::OtherCompilationError;
}
}
if (!!(Flags & EHotReloadFlags::WaitForCompletion))
{
Ar.Logf(ELogVerbosity::Warning, TEXT("HotReload operation took %4.1fs."), float(FPlatformTime::Seconds() - HotReloadStartTime));
bIsHotReloadingFromEditor = false;
}
else
{
Ar.Logf(ELogVerbosity::Warning, TEXT("Starting HotReload took %4.1fs."), float(FPlatformTime::Seconds() - HotReloadStartTime));
}
return ECompilationResult::Succeeded;
#else
Ar.Logf(ELogVerbosity::Warning, TEXT("RebindPackages not possible (hot reload not supported)"));
return ECompilationResult::Unsupported;
#endif
}
#if WITH_ENGINE
namespace {
static TArray<TPair<UClass*, UClass*> >& GetClassesToReinstance()
{
static TArray<TPair<UClass*, UClass*> > Data;
return Data;
}
}
void FHotReloadModule::RegisterForReinstancing(UClass* OldClass, UClass* NewClass)
{
TPair<UClass*, UClass*> Pair;
Pair.Key = OldClass;
Pair.Value = NewClass;
TArray<TPair<UClass*, UClass*> >& ClassesToReinstance = GetClassesToReinstance();
ClassesToReinstance.Add(MoveTemp(Pair));
}
void FHotReloadModule::ReinstanceClasses()
{
#if WITH_HOT_RELOAD
if (GIsHotReload)
{
UClass::AssembleReferenceTokenStreams();
}
#endif // WITH_HOT_RELOAD
TArray<TPair<UClass*, UClass*> >& ClassesToReinstance = GetClassesToReinstance();
TMap<UClass*, UClass*> OldToNewClassesMap;
for (const TPair<UClass*, UClass*>& Pair : ClassesToReinstance)
{
// Don't allow reinstancing of UEngine classes
if (Pair.Key->IsChildOf(UEngine::StaticClass()))
{
UE_LOG(LogHotReload, Warning, TEXT("Engine class '%s' has changed but will be ignored for hot reload"), *Pair.Key->GetName());
continue;
}
if (Pair.Value != nullptr)
{
OldToNewClassesMap.Add(Pair.Key, Pair.Value);
}
}
for (const TPair<UClass*, UClass*>& Pair : ClassesToReinstance)
{
// Don't allow reinstancing of UEngine classes
if (!Pair.Key->IsChildOf(UEngine::StaticClass()))
{
ReinstanceClass(Pair.Key, Pair.Value, OldToNewClassesMap);
}
}
ClassesToReinstance.Empty();
}
void FHotReloadModule::ReinstanceClass(UClass* OldClass, UClass* NewClass, const TMap<UClass*, UClass*>& OldToNewClassesMap)
{
TSharedPtr<FHotReloadClassReinstancer> ReinstanceHelper = FHotReloadClassReinstancer::Create(NewClass, OldClass, OldToNewClassesMap, ReconstructedCDOsMap, HotReloadBPSetToRecompile, HotReloadBPSetToRecompileBytecodeOnly);
if (ReinstanceHelper->ClassNeedsReinstancing())
{
UE_LOG(LogHotReload, Log, TEXT("Re-instancing %s after hot-reload."), NewClass ? *NewClass->GetName() : *OldClass->GetName());
ReinstanceHelper->ReinstanceObjectsAndUpdateDefaults();
}
}
#endif
void FHotReloadModule::OnHotReloadBinariesChanged(const TArray<FFileChangeData>& FileChanges)
{
if (bIsHotReloadingFromEditor)
{
// DO NOTHING, this case is handled by RebindPackages
return;
}
const FModuleManager& ModuleManager = FModuleManager::Get();
TMap<FString, FString> GameModuleFilenames = UE4HotReload_Private::GetGameModuleFilenames(ModuleManager);
if (GameModuleFilenames.Num() == 0)
{
return;
}
// Check if any of the game DLLs has been added
for (const FFileChangeData& Change : FileChanges)
{
// Ignore changes that aren't introducing a new file.
//
// On the Mac the Add event is for a temporary linker(?) file that gets immediately renamed
// to a dylib. In the future we may want to support modified event for all platforms anyway once
// shadow copying works with hot-reload.
#if PLATFORM_MAC
if (Change.Action != FFileChangeData::FCA_Modified)
#else
if (Change.Action != FFileChangeData::FCA_Added)
#endif
{
continue;
}
// Ignore files that aren't of module type
FString Filename = FPaths::GetCleanFilename(Change.Filename);
if (!Filename.EndsWith(FPlatformProcess::GetModuleExtension()))
{
continue;
}
for (const TPair<FString, FString>& NameFilename : GameModuleFilenames)
{
// Handle module files which have already been hot-reloaded.
FString BaseName = FPaths::GetBaseFilename(NameFilename.Value);
StripModuleSuffixFromFilename(BaseName, NameFilename.Key);
// Hot reload always adds a numbered suffix preceded by a hyphen, but otherwise the module name must match exactly!
if (!Filename.StartsWith(BaseName + TEXT("-")))
{
continue;
}
if (ModulesRecentlyCompiledInTheEditor.Contains(FPaths::ConvertRelativePathToFull(Change.Filename)))
{
continue;
}
// Add to queue. We do not hot-reload here as there may potentially be other modules being compiled.
DetectedNewModules.Emplace(NameFilename.Key, Change.Filename);
UE_LOG(LogHotReload, Log, TEXT("New module detected: %s"), *Filename);
}
}
}
void FHotReloadModule::StripModuleSuffixFromFilename(FString& InOutModuleFilename, const FString& ModuleName)
{
// First hyphen is where the UE4Edtior prefix ends
int32 FirstHyphenIndex = INDEX_NONE;
if (InOutModuleFilename.FindChar('-', FirstHyphenIndex))
{
// Second hyphen means we already have a hot-reloaded module or other than Development config module
int32 SecondHyphenIndex = FirstHyphenIndex;
do
{
SecondHyphenIndex = InOutModuleFilename.Find(TEXT("-"), ESearchCase::IgnoreCase, ESearchDir::FromStart, SecondHyphenIndex + 1);
if (SecondHyphenIndex != INDEX_NONE)
{
// Make sure that the section between hyphens is the expected module name. This guards against cases where module name has a hyphen inside.
FString HotReloadedModuleName = InOutModuleFilename.Mid(FirstHyphenIndex + 1, SecondHyphenIndex - FirstHyphenIndex - 1);
if (HotReloadedModuleName == ModuleName)
{
InOutModuleFilename = InOutModuleFilename.Mid(0, SecondHyphenIndex);
SecondHyphenIndex = INDEX_NONE;
}
}
} while (SecondHyphenIndex != INDEX_NONE);
}
}
void FHotReloadModule::RefreshHotReloadWatcher()
{
FDirectoryWatcherModule& DirectoryWatcherModule = FModuleManager::Get().LoadModuleChecked<FDirectoryWatcherModule>(TEXT("DirectoryWatcher"));
IDirectoryWatcher* DirectoryWatcher = DirectoryWatcherModule.Get();
if (DirectoryWatcher)
{
// Watch the game directory
AddHotReloadDirectory(DirectoryWatcher, FPaths::ProjectDir());
// Also watch all the game plugin directories
for(const TSharedRef<IPlugin>& Plugin : IPluginManager::Get().GetEnabledPlugins())
{
if (Plugin->GetLoadedFrom() == EPluginLoadedFrom::Project && Plugin->GetDescriptor().Modules.Num() > 0)
{
AddHotReloadDirectory(DirectoryWatcher, Plugin->GetBaseDir());
}
}
}
}
void FHotReloadModule::AddHotReloadDirectory(IDirectoryWatcher* DirectoryWatcher, const FString& BaseDir)
{
FString BinariesPath = FPaths::ConvertRelativePathToFull(BaseDir / TEXT("Binaries") / FPlatformProcess::GetBinariesSubdirectory());
if (FPaths::DirectoryExists(BinariesPath) && !BinariesFolderChangedDelegateHandles.Contains(BinariesPath))
{
IDirectoryWatcher::FDirectoryChanged BinariesFolderChangedDelegate = IDirectoryWatcher::FDirectoryChanged::CreateRaw(this, &FHotReloadModule::OnHotReloadBinariesChanged);
FDelegateHandle Handle;
if (DirectoryWatcher->RegisterDirectoryChangedCallback_Handle(BinariesPath, BinariesFolderChangedDelegate, Handle))
{
BinariesFolderChangedDelegateHandles.Add(BinariesPath, Handle);
}
}
}
void FHotReloadModule::ShutdownHotReloadWatcher()
{
FDirectoryWatcherModule* DirectoryWatcherModule = FModuleManager::GetModulePtr<FDirectoryWatcherModule>(TEXT("DirectoryWatcher"));
if( DirectoryWatcherModule != nullptr )
{
IDirectoryWatcher* DirectoryWatcher = DirectoryWatcherModule->Get();
if (DirectoryWatcher)
{
for (const TPair<FString, FDelegateHandle>& Pair : BinariesFolderChangedDelegateHandles)
{
DirectoryWatcher->UnregisterDirectoryChangedCallback_Handle(Pair.Key, Pair.Value);
}
}
}
}
bool FHotReloadModule::Tick(float DeltaTime)
{
// We never want to block on a pending compile when checking compilation status during Tick(). We're
// just checking so that we can fire callbacks if and when compilation has finished.
// Ignored output variables
bool bCompileStillInProgress = false;
bool bCompileSucceeded = false;
FOutputDeviceNull NullOutput;
CheckForFinishedModuleDLLCompile( EHotReloadFlags::None, bCompileStillInProgress, bCompileSucceeded, NullOutput );
if (DetectedNewModules.Num() == 0)
{
return true;
}
#if WITH_EDITOR
if (GEditor)
{
// Don't try to do an IDE reload yet if we're PIE - wait until we leave
if (GEditor->bIsPlayWorldQueued || GEditor->PlayWorld)
{
return true;
}
// Don't allow hot reloading if we're running networked PIE instances
// The reason, is it's fairly complicated to handle the re-wiring that needs to happen when we re-instance objects like player controllers, possessed pawns, etc...
const TIndirectArray<FWorldContext>& WorldContextList = GEditor->GetWorldContexts();
for (const FWorldContext& WorldContext : WorldContextList)
{
if (WorldContext.World() && WorldContext.World()->WorldType == EWorldType::PIE && WorldContext.World()->NetDriver)
{
return true; // Don't allow automatic hot reloading if we're running PIE instances
}
}
}
#endif // WITH_EDITOR
// We have new modules in the queue, but make sure UBT has finished compiling all of them
if (!FDesktopPlatformModule::Get()->IsUnrealBuildToolRunning())
{
IFileManager& FileManager = IFileManager::Get();
// Remove any modules whose files have disappeared - this can happen if a compile event has
// failed and deleted a DLL that was there previously.
for (auto It = DetectedNewModules.CreateIterator(); It; ++It)
{
if (!FileManager.FileExists(*It->Value))
{
It.RemoveCurrent();
}
}
DoHotReloadFromIDE(DetectedNewModules);
DetectedNewModules.Empty();
}
else
{
UE_LOG(LogHotReload, Verbose, TEXT("Detected %d reloaded modules but UnrealBuildTool is still running"), DetectedNewModules.Num());
}
return true;
}
void FHotReloadModule::DoHotReloadFromIDE(const TMap<FString, FString>& NewModules)
{
const FModuleManager& ModuleManager = FModuleManager::Get();
int32 NumPackagesToRebind = 0;
int32 NumDependentModules = 0;
ECompilationResult::Type Result = ECompilationResult::Unsupported;
double Duration = 0.0;
TArray<FString> GameModuleNames = UE4HotReload_Private::GetGameModuleNames(ModuleManager);
if (GameModuleNames.Num() > 0)
{
FScopedDurationTimer Timer(Duration);
if (NewModules.Num() == 0)
{
return;
}
UE_LOG(LogHotReload, Log, TEXT("Starting Hot-Reload from IDE"));
HotReloadStartTime = FPlatformTime::Seconds();
FScopedSlowTask SlowTask(100.f, LOCTEXT("CompilingGameCode", "Compiling Game Code"));
SlowTask.MakeDialog();
// Update compile data before we start compiling
for (const TPair<FString, FString>& NewModule : NewModules)
{
// Move on 10% / num items
SlowTask.EnterProgressFrame(10.f/NewModules.Num());
FName ModuleName = *NewModule.Key;
UpdateModuleCompileData(ModuleName);
OnModuleCompileSucceeded(ModuleName, NewModule.Value);
}
SlowTask.EnterProgressFrame(10);
UE4HotReload_Private::FPackagesAndDependentNames PackagesAndDependentNames = UE4HotReload_Private::SplitByPackagesAndDependentNames(GameModuleNames);
SlowTask.EnterProgressFrame(80);
NumPackagesToRebind = PackagesAndDependentNames.Packages.Num();
NumDependentModules = PackagesAndDependentNames.DependentNames.Num();
Result = DoHotReloadInternal(NewModules, PackagesAndDependentNames.Packages, PackagesAndDependentNames.DependentNames, *GLog);
}
RecordAnalyticsEvent(TEXT("IDE"), Result, Duration, NumPackagesToRebind, NumDependentModules);
}
void FHotReloadModule::RecordAnalyticsEvent(const TCHAR* ReloadFrom, ECompilationResult::Type Result, double Duration, int32 PackageCount, int32 DependentModulesCount)
{
#if WITH_ENGINE
if (FEngineAnalytics::IsAvailable())
{
TArray< FAnalyticsEventAttribute > ReloadAttribs;
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("ReloadFrom"), ReloadFrom));
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("Result"), ECompilationResult::ToString(Result)));
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("Duration"), FString::Printf(TEXT("%.4lf"), Duration)));
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("Packages"), FString::Printf(TEXT("%d"), PackageCount)));
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("DependentModules"), FString::Printf(TEXT("%d"), DependentModulesCount)));
FEngineAnalytics::GetProvider().RecordEvent(TEXT("Editor.Usage.HotReload"), ReloadAttribs);
}
#endif
}
void FHotReloadModule::OnModuleCompileSucceeded(FName ModuleName, const FString& ModuleFilename)
{
// If the compile succeeded, update the module info entry with the new file name for this module
FModuleManager::Get().SetModuleFilename(ModuleName, ModuleFilename);
#if WITH_HOT_RELOAD
// UpdateModuleCompileData() should have been run before compiling so the
// data in ModuleInfo should be correct for the pre-compile dll file.
FModuleCompilationData& CompileData = ModuleCompileData.FindChecked(ModuleName).Get();
FDateTime FileTimeStamp;
bool bGotFileTimeStamp = GetModuleFileTimeStamp(ModuleName, FileTimeStamp);
CompileData.bHasFileTimeStamp = bGotFileTimeStamp;
CompileData.FileTimeStamp = FileTimeStamp;
if (CompileData.bHasFileTimeStamp)
{
CompileData.CompileMethod = EModuleCompileMethod::Runtime;
}
else
{
CompileData.CompileMethod = EModuleCompileMethod::Unknown;
}
WriteModuleCompilationInfoToConfig(ModuleName, CompileData);
#endif
}
FString FHotReloadModule::MakeUBTArgumentsForModuleCompiling()
{
FString AdditionalArguments;
if ( FPaths::IsProjectFilePathSet() )
{
// We have to pass FULL paths to UBT
FString FullProjectPath = FPaths::ConvertRelativePathToFull(FPaths::GetProjectFilePath());
// @todo projectdirs: Currently non-installed projects that exist under the UE4 root are compiled by UBT with no .uproject file
// name passed in (see bIsProjectTarget in VCProject.cs), which causes intermediate libraries to be saved to the Engine
// intermediate folder instead of the project's intermediate folder. We're emulating this behavior here for module
// recompiling, so that compiled modules will be able to find their import libraries in the original folder they were compiled.
if( FApp::IsEngineInstalled() || !FullProjectPath.StartsWith( FPaths::ConvertRelativePathToFull( FPaths::RootDir() ) ) )
{
const FString ProjectFilenameWithQuotes = FString::Printf(TEXT("\"%s\""), *FullProjectPath);
AdditionalArguments += FString::Printf(TEXT("%s "), *ProjectFilenameWithQuotes);
}
}
// Use new FastPDB option to cut down linking time. Currently disabled due to problems with missing symbols in VS2015.
// AdditionalArguments += TEXT(" -FastPDB");
return AdditionalArguments;
}
#if WITH_HOT_RELOAD
bool FHotReloadModule::StartCompilingModuleDLLs(const FString& GameName, const TArray< FModuleToRecompile >& ModuleNames,
FRecompileModulesCallback&& InRecompileModulesCallback, FOutputDevice& Ar, bool bInFailIfGeneratedCodeChanges,
const FString& InAdditionalCmdLineArgs, bool bForceCodeProject)
{
// Keep track of what we're compiling
ModulesBeingCompiled = ModuleNames;
ModulesThatWereBeingRecompiled = ModulesBeingCompiled;
const TCHAR* BuildPlatformName = FPlatformMisc::GetUBTPlatform();
const TCHAR* BuildConfigurationName = FModuleManager::GetUBTConfiguration();
RecompileModulesCallback = MoveTemp(InRecompileModulesCallback);
// Pass a module file suffix to UBT if we have one
FString ModuleArg;
if (ModuleNames.Num())
{
Ar.Logf(TEXT("Candidate modules for hot reload:"));
for( const FModuleToRecompile& Module : ModuleNames )
{
if( !Module.ModuleFileSuffix.IsEmpty() )
{
ModuleArg += FString::Printf( TEXT( " -ModuleWithSuffix %s %s" ), *Module.ModuleName, *Module.ModuleFileSuffix );
}
else
{
ModuleArg += FString::Printf( TEXT( " -Module %s" ), *Module.ModuleName );
}
Ar.Logf( TEXT( " %s" ), *Module.ModuleName );
// prepare the compile info in the FModuleInfo so that it can be compared after compiling
FName ModuleFName(*Module.ModuleName);
UpdateModuleCompileData(ModuleFName);
}
}
FString ExtraArg;
#if UE_EDITOR
// NOTE: When recompiling from the editor, we're passed the game target name, not the editor target name, but we'll
// pass "-editorrecompile" to UBT which tells UBT to figure out the editor target to use for this game, since
// we can't possibly know what the target is called from within the engine code.
ExtraArg = TEXT( "-editorrecompile " );
#endif
if (bInFailIfGeneratedCodeChanges)
{
// Additional argument to let UHT know that we can only compile the module if the generated code didn't change
ExtraArg += TEXT( "-FailIfGeneratedCodeChanges " );
}
// If there's nothing to compile, don't bother linking the DLLs as the old ones are up-to-date
ExtraArg += TEXT("-canskiplink ");
// Shared PCH does no work with hot-reloading engine/editor modules as we don't scan all modules for them.
if (!ContainsOnlyGameModules(ModuleNames))
{
ExtraArg += TEXT("-nosharedpch ");
}
FString TargetName = GameName;
#if WITH_EDITOR
// If there are no game modules loaded, then it's not a code-based project and the target
// for UBT should be the editor.
if (!bForceCodeProject && !IsAnyGameModuleLoaded())
{
TargetName = TEXT("UE4Editor");
}
#endif
FString CmdLineParams = FString::Printf( TEXT( "%s%s %s %s %s%s" ),
*TargetName, *ModuleArg,
BuildPlatformName, BuildConfigurationName,
*ExtraArg, *InAdditionalCmdLineArgs );
const bool bInvocationSuccessful = InvokeUnrealBuildToolForCompile(CmdLineParams, Ar);
if ( !bInvocationSuccessful )
{
// No longer compiling modules
ModulesBeingCompiled.Empty();
ModuleCompilerFinishedEvent.Broadcast(FString(), ECompilationResult::OtherCompilationError, false);
// Fire task completion delegate
if (RecompileModulesCallback)
{
RecompileModulesCallback( TMap<FString, FString>(), false, ECompilationResult::OtherCompilationError );
RecompileModulesCallback = nullptr;
}
}
return bInvocationSuccessful;
}
#endif
bool FHotReloadModule::InvokeUnrealBuildToolForCompile(const FString& InCmdLineParams, FOutputDevice &Ar)
{
#if WITH_HOT_RELOAD
// Make sure we're not already compiling something!
check(!IsCurrentlyCompiling());
// Setup output redirection pipes, so that we can harvest compiler output and display it ourselves
void* PipeRead = NULL;
void* PipeWrite = NULL;
verify(FPlatformProcess::CreatePipe(PipeRead, PipeWrite));
ModuleCompileReadPipeText = TEXT("");
FProcHandle ProcHandle = FDesktopPlatformModule::Get()->InvokeUnrealBuildToolAsync(InCmdLineParams, Ar, PipeRead, PipeWrite);
// We no longer need the Write pipe so close it.
// We DO need the Read pipe however...
FPlatformProcess::ClosePipe(0, PipeWrite);
if (!ProcHandle.IsValid())
{
// We're done with the process handle now
ModuleCompileProcessHandle.Reset();
ModuleCompileReadPipe = NULL;
}
else
{
ModuleCompileProcessHandle = ProcHandle;
ModuleCompileReadPipe = PipeRead;
}
return ProcHandle.IsValid();
#else
return false;
#endif // WITH_HOT_RELOAD
}
void FHotReloadModule::CheckForFinishedModuleDLLCompile(EHotReloadFlags Flags, bool& bCompileStillInProgress, bool& bCompileSucceeded, FOutputDevice& Ar, bool bFireEvents)
{
#if WITH_HOT_RELOAD
bCompileStillInProgress = false;
ECompilationResult::Type CompilationResult = ECompilationResult::OtherCompilationError;
// Is there a compilation in progress?
if( !IsCurrentlyCompiling() )
{
Ar.Logf(TEXT("Error: CheckForFinishedModuleDLLCompile: There is no compilation in progress right now"));
return;
}
bCompileStillInProgress = true;
FText StatusUpdate;
if ( ModulesBeingCompiled.Num() > 0 )
{
FFormatNamedArguments Args;
Args.Add( TEXT("CodeModuleName"), FText::FromString( ModulesBeingCompiled[0].ModuleName ) );
StatusUpdate = FText::Format( NSLOCTEXT("FModuleManager", "CompileSpecificModuleStatusMessage", "{CodeModuleName}: Compiling modules..."), Args );
}
else
{
StatusUpdate = NSLOCTEXT("FModuleManager", "CompileStatusMessage", "Compiling modules...");
}
FScopedSlowTask SlowTask(0, StatusUpdate, GIsSlowTask);
SlowTask.MakeDialog();
// Check to see if the compile has finished yet
int32 ReturnCode = -1;
while (bCompileStillInProgress)
{
// Store the return code in a temp variable for now because it still gets overwritten
// when the process is running.
int32 ProcReturnCode = -1;
if( FPlatformProcess::GetProcReturnCode( ModuleCompileProcessHandle, &ProcReturnCode ) )
{
ReturnCode = ProcReturnCode;
bCompileStillInProgress = false;
}
if (bRequestCancelCompilation)
{
FPlatformProcess::TerminateProc(ModuleCompileProcessHandle);
bCompileStillInProgress = bRequestCancelCompilation = false;
}
if( bCompileStillInProgress )
{
ModuleCompileReadPipeText += FPlatformProcess::ReadPipe(ModuleCompileReadPipe);
if (!(Flags & EHotReloadFlags::WaitForCompletion))
{
// We haven't finished compiling, but we were asked to return immediately
break;
}
SlowTask.EnterProgressFrame(0.0f);
// Give up a small timeslice if we haven't finished recompiling yet
FPlatformProcess::Sleep( 0.01f );
}
}
bRequestCancelCompilation = false;
if( bCompileStillInProgress )
{
Ar.Logf(TEXT("Error: CheckForFinishedModuleDLLCompile: Compilation is still in progress"));
return;
}
// Compilation finished, now we need to grab all of the text from the output pipe
ModuleCompileReadPipeText += FPlatformProcess::ReadPipe(ModuleCompileReadPipe);
// This includes 'canceled' (-1) and 'up-to-date' (-2)
CompilationResult = (ECompilationResult::Type)ReturnCode;
// If compilation succeeded for all modules, go back to the modules and update their module file names
// in case we recompiled the modules to a new unique file name. This is needed so that when the module
// is reloaded after the recompile, we load the new DLL file name, not the old one.
// Note that we don't want to do anything in case the build was canceled or source code has not changed.
TMap<FString, FString> ChangedModules;
if(CompilationResult == ECompilationResult::Succeeded)
{
ChangedModules.Reserve(ModulesThatWereBeingRecompiled.Num());
for( FModuleToRecompile& CurModule : ModulesThatWereBeingRecompiled )
{
// Were we asked to assign a new file name for this module?
if( CurModule.NewModuleFilename.IsEmpty() )
{
continue;
}
if (IFileManager::Get().FileSize(*CurModule.NewModuleFilename) <= 0)
{
continue;
}
// If the file doesn't exist, then assume it doesn't needs rebinding because it wasn't recompiled
FDateTime FileTimeStamp = IFileManager::Get().GetTimeStamp(*CurModule.NewModuleFilename);
if (FileTimeStamp == FDateTime::MinValue())
{
continue;
}
FName ModuleName = *CurModule.ModuleName;
// If the file is the same as what we remembered it was then assume it doesn't needs rebinding because it wasn't recompiled
TSharedRef<FModuleCompilationData>* CompileDataPtr = ModuleCompileData.Find(ModuleName);
if (CompileDataPtr && (*CompileDataPtr)->FileTimeStamp == FileTimeStamp)
{
continue;
}
// If the compile succeeded, update the module info entry with the new file name for this module
OnModuleCompileSucceeded(ModuleName, CurModule.NewModuleFilename);
// Move modules
ChangedModules.Emplace(MoveTemp(CurModule.ModuleName), MoveTemp(CurModule.NewModuleFilename));
}
}
ModulesThatWereBeingRecompiled.Empty();
// We're done with the process handle now
FPlatformProcess::CloseProc(ModuleCompileProcessHandle);
ModuleCompileProcessHandle.Reset();
FPlatformProcess::ClosePipe(ModuleCompileReadPipe, 0);
Ar.Log(*ModuleCompileReadPipeText);
const FString FinalOutput = ModuleCompileReadPipeText;
ModuleCompileReadPipe = NULL;
ModuleCompileReadPipeText = TEXT("");
// No longer compiling modules
ModulesBeingCompiled.Empty();
bCompileSucceeded = !ECompilationResult::Failed(CompilationResult);
if ( bFireEvents )
{
const bool bShowLogOnSuccess = false;
ModuleCompilerFinishedEvent.Broadcast(FinalOutput, CompilationResult, !bCompileSucceeded || bShowLogOnSuccess);
// Fire task completion delegate
if (RecompileModulesCallback)
{
RecompileModulesCallback( ChangedModules, true, CompilationResult );
RecompileModulesCallback = nullptr;
}
}
#endif // WITH_HOT_RELOAD
}
void FHotReloadModule::UpdateModuleCompileData(FName ModuleName)
{
// Find or create a compile data object for this module
TSharedRef<FModuleCompilationData>* CompileDataPtr = ModuleCompileData.Find(ModuleName);
if(CompileDataPtr == nullptr)
{
CompileDataPtr = &ModuleCompileData.Add(ModuleName, TSharedRef<FModuleCompilationData>(new FModuleCompilationData()));
}
// reset the compile data before updating it
FModuleCompilationData& CompileData = CompileDataPtr->Get();
CompileData.bHasFileTimeStamp = false;
CompileData.FileTimeStamp = FDateTime(0);
CompileData.CompileMethod = EModuleCompileMethod::Unknown;
#if WITH_HOT_RELOAD
ReadModuleCompilationInfoFromConfig(ModuleName, CompileData);
FDateTime FileTimeStamp;
bool bGotFileTimeStamp = GetModuleFileTimeStamp(ModuleName, FileTimeStamp);
if (!bGotFileTimeStamp)
{
// File missing? Reset the cached timestamp and method to defaults and save them.
CompileData.bHasFileTimeStamp = false;
CompileData.FileTimeStamp = FDateTime(0);
CompileData.CompileMethod = EModuleCompileMethod::Unknown;
WriteModuleCompilationInfoToConfig(ModuleName, CompileData);
}
else
{
if (CompileData.bHasFileTimeStamp)
{
if (FileTimeStamp > CompileData.FileTimeStamp + HotReloadDefs::TimeStampEpsilon)
{
// The file is newer than the cached timestamp
// The file must have been compiled externally
CompileData.FileTimeStamp = FileTimeStamp;
CompileData.CompileMethod = EModuleCompileMethod::External;
WriteModuleCompilationInfoToConfig(ModuleName, CompileData);
}
}
else
{
// The cached timestamp and method are default value so this file has no history yet
// We can only set its timestamp and save
CompileData.bHasFileTimeStamp = true;
CompileData.FileTimeStamp = FileTimeStamp;
WriteModuleCompilationInfoToConfig(ModuleName, CompileData);
}
}
#endif
}
void FHotReloadModule::ReadModuleCompilationInfoFromConfig(FName ModuleName, FModuleCompilationData& CompileData)
{
FString DateTimeString;
if (GConfig->GetString(*HotReloadDefs::CompilationInfoConfigSection, *FString::Printf(TEXT("%s.TimeStamp"), *ModuleName.ToString()), DateTimeString, GEditorPerProjectIni))
{
FDateTime TimeStamp;
if (!DateTimeString.IsEmpty() && FDateTime::Parse(DateTimeString, TimeStamp))
{
CompileData.bHasFileTimeStamp = true;
CompileData.FileTimeStamp = TimeStamp;
FString CompileMethodString;
if (GConfig->GetString(*HotReloadDefs::CompilationInfoConfigSection, *FString::Printf(TEXT("%s.LastCompileMethod"), *ModuleName.ToString()), CompileMethodString, GEditorPerProjectIni))
{
if (CompileMethodString.Equals(HotReloadDefs::CompileMethodRuntime, ESearchCase::IgnoreCase))
{
CompileData.CompileMethod = EModuleCompileMethod::Runtime;
}
else if (CompileMethodString.Equals(HotReloadDefs::CompileMethodExternal, ESearchCase::IgnoreCase))
{
CompileData.CompileMethod = EModuleCompileMethod::External;
}
}
}
}
}
void FHotReloadModule::WriteModuleCompilationInfoToConfig(FName ModuleName, const FModuleCompilationData& CompileData)
{
FString DateTimeString;
if (CompileData.bHasFileTimeStamp)
{
DateTimeString = CompileData.FileTimeStamp.ToString();
}
GConfig->SetString(*HotReloadDefs::CompilationInfoConfigSection, *FString::Printf(TEXT("%s.TimeStamp"), *ModuleName.ToString()), *DateTimeString, GEditorPerProjectIni);
FString CompileMethodString = HotReloadDefs::CompileMethodUnknown;
if (CompileData.CompileMethod == EModuleCompileMethod::Runtime)
{
CompileMethodString = HotReloadDefs::CompileMethodRuntime;
}
else if (CompileData.CompileMethod == EModuleCompileMethod::External)
{
CompileMethodString = HotReloadDefs::CompileMethodExternal;
}
GConfig->SetString(*HotReloadDefs::CompilationInfoConfigSection, *FString::Printf(TEXT("%s.LastCompileMethod"), *ModuleName.ToString()), *CompileMethodString, GEditorPerProjectIni);
}
bool FHotReloadModule::GetModuleFileTimeStamp(FName ModuleName, FDateTime& OutFileTimeStamp) const
{
FString Filename = FModuleManager::Get().GetModuleFilename(ModuleName);
if (IFileManager::Get().FileSize(*Filename) > 0)
{
OutFileTimeStamp = FDateTime(IFileManager::Get().GetTimeStamp(*Filename));
return true;
}
return false;
}
bool FHotReloadModule::IsAnyGameModuleLoaded()
{
if (bIsAnyGameModuleLoaded == EThreeStateBool::Unknown)
{
bool bGameModuleFound = false;
// Ask the module manager for a list of currently-loaded gameplay modules
TArray< FModuleStatus > ModuleStatuses;
FModuleManager::Get().QueryModules(ModuleStatuses);
for (const FModuleStatus& ModuleStatus : ModuleStatuses)
{
// We only care about game modules that are currently loaded
if (ModuleStatus.bIsLoaded && ModuleStatus.bIsGameModule)
{
// There is at least one loaded game module.
bGameModuleFound = true;
break;
}
}
bIsAnyGameModuleLoaded = EThreeStateBool::FromBool(bGameModuleFound);
}
return EThreeStateBool::ToBool(bIsAnyGameModuleLoaded);
}
bool FHotReloadModule::ContainsOnlyGameModules(const TArray<FModuleToRecompile>& ModulesToCompile) const
{
FString AbsoluteProjectDir = FPaths::ConvertRelativePathToFull(FPaths::ProjectDir());
for (const FModuleToRecompile& ModuleToCompile : ModulesToCompile)
{
FString FullModulePath = FPaths::ConvertRelativePathToFull(ModuleToCompile.NewModuleFilename);
if (!FullModulePath.StartsWith(AbsoluteProjectDir))
{
return false;
}
}
return true;
}
void FHotReloadModule::ModulesChangedCallback(FName ModuleName, EModuleChangeReason ReasonForChange)
{
// Force update game modules state on the next call to IsAnyGameModuleLoaded
bIsAnyGameModuleLoaded = EThreeStateBool::Unknown;
// If the hot reload directory watcher hasn't been initialized yet (because the binaries directory did not exist) try to initialize it now
if (!bDirectoryWatcherInitialized)
{
RefreshHotReloadWatcher();
bDirectoryWatcherInitialized = true;
}
}
void FHotReloadModule::PluginMountedCallback(IPlugin& Plugin)
{
FDirectoryWatcherModule& DirectoryWatcherModule = FModuleManager::Get().LoadModuleChecked<FDirectoryWatcherModule>(TEXT("DirectoryWatcher"));
IDirectoryWatcher* DirectoryWatcher = DirectoryWatcherModule.Get();
if (DirectoryWatcher)
{
if (Plugin.GetLoadedFrom() == EPluginLoadedFrom::Project && Plugin.GetDescriptor().Modules.Num() > 0)
{
AddHotReloadDirectory(DirectoryWatcher, Plugin.GetBaseDir());
}
}
}
#undef LOCTEXT_NAMESPACE