Files
UnrealEngineUWP/Engine/Source/Developer/HotReload/Private/HotReload.cpp
Matt Kuhlenschmidt c72e1e1e70 Copying //UE4/Dev-Editor to //UE4/Dev-Main (Source: //UE4/Dev-Editor @ 3739701)
#lockdown Nick.Penwarden
#rb none

============================
  MAJOR FEATURES & CHANGES
============================

Change 3358367 by tim.gautier

	Submitting resaved QAGame assets - Materials, Material Instances, Material Functions and Parameters

Change 3624848 by Jamie.Dale

	Added a composite font for the editor (and Slate core)

	This is defined in FLegacySlateFontInfoCache::GetDefaultFont and uses our default Roboto fonts (and the culture specific fallback fonts), and is now used as the default font for Slate and the editor.

	This change removes all the manual TTF/OTF file references from the various Slate styles, as well as updating 200+ hard-coded font references to use the new default font.

	This fixes various rendering issues with fonts in the editor when using different languages, and clears a big barrier for removing the legacy localized fallback font support.


Change 3654993 by Jamie.Dale

	'Native' (now called 'FNativeFuncPtr') is now a function pointer that takes a UObject* context, rather than a UObject member function pointer

	This avoids ambiguity when binding a native function pointer to a type that doesn't match the context pointer, as you could end up getting a function called with an incorrect 'this' pointer

	Breaking changes:
	 - Native has been renamed to FNativeFuncPtr.
	 - The signature of a native function has changed (use the DECLARE_FUNCTION and DEFINE_FUNCTION macro pair).
	 - Use P_THIS if you were previously using the 'this' pointer in your native function.

Change 3699591 by Jamie.Dale

	Added support for displaying and editing numbers in a culture correct way

	Numeric input boxes in Slate will now display and accept numbers using the culture correct decimal separators. This is enabled by default, and can be disabled by setting "ShouldUseLocalizedNumericInput" to "False" in XEditorSettings.ini (for the editor), or XGameUserSettings.ini (for a game).

	#jira UE-4028


Change 3719568 by Jamie.Dale

	Allow platforms to override the default ICU timezone calculation


Change 3622366 by Bradut.Palas

	#jira UE-46677

	Don't allow OnLevelRemovedFromWorld to reset the transaction buffer if we're in PIE mode.
	Also, remove one undo barrier in case the event was triggered in PIE mode or else we block the user from undoing previous actions.

Change 3622378 by Bradut.Palas

	#jira UE-46590

	we have a general bug with detecting the size of the last column, but the clamping prevents it from appearing with the other resize modes. The Content Browser is the only one to use fixed width.
	The bug is that the size of the last element is incorrectly reported, after we drag back and forth.
	Fixed by not reading the size real time, but reading it from the SlotInfo structure that is created earlier, which holds the correct value.

Change 3622552 by Jamie.Dale

	Added support for per-culture sub-fonts within a composite font

	This allows you to do things like create a Japanese specific Han sub-font to override the Han characters used in a CJK font (previously you needed to create a localized font asset to achieve this).

Change 3623170 by Jamie.Dale

	Fixing warning

Change 3624846 by Jamie.Dale

	Composite font cache optimizations

	- Converted a typically small sized map to a sorted array + binary search.
	- Converted the already sorted range array to use binary search.
	- Contiguous ranges using the same typeface are now merged in the cache.

Change 3625576 by Cody.Albert

	We now only set the widget tree to transient instead of passing the flag through StaticDuplicateObject. This was causing instanced subobjects to be flagged with RF_DuplicateTransient, preventing them from properly being duplicated when an array of instanced subobjects was modified.

	#jira UE-47971

Change 3626057 by Matt.Kuhlenschmidt

	Expose EUmgSequencePlayMode to blueprints

	#jira UE-49255

Change 3626556 by Matt.Kuhlenschmidt

	Fix window size and position adjustment not accounting for primary monitor not being a high DPI monitor when a secondary monitor is.  Causes flickering and incorrect window positioning.

	#jira UE-48922, UE-48957

Change 3627692 by Matt.Kuhlenschmidt

	PR #3977: Source control submenu menu customization (Contributed by Kryofenix)


Change 3628600 by Arciel.Rekman

	Added AutoCheckout to FAssetRenameManager for commandlet usage.

Change 3630561 by Richard.Hinckley

	Deprecating the version of UFunctionalTestingManager::RunAllFunctionalTests that feature an unused bool parameter, replacing with a new version without that parameter.

Change 3630656 by Richard.Hinckley

	Compile fix.

Change 3630964 by Arciel.Rekman

	Fix CrashReporterClient headless build.

Change 3631050 by Matt.Kuhlenschmidt

	Back out revision 9 from //UE4/Dev-Editor/Engine/Source/Runtime/Slate/Private/Widgets/Layout/SSplitter.cpp

	Causes major problems with resizing splitters in editor

Change 3631140 by Arciel.Rekman

	OpenAL: update Linux version to 1.18.1 (UETOOL-1253)

	- Also remove a hack for RPATH and make it use a generic RPATH mechanism.
	- Bulk of the change from Cengiz.Terzibas

	#jira UETOOL-1253

Change 3632924 by Jamie.Dale

	Added support for a catch-all fallback font within composite fonts

	This allows you to provide broad "font of last resort" behavior on a per-composite font basis, in a way that can also work with different font styles.

Change 3633055 by Jamie.Dale

	Fixed some refresh issues in the font editor

Change 3633062 by Jamie.Dale

	Fixed localization commands being reported as unknown

Change 3633906 by Nick.Darnell

	UMG - You can now store refrences to widgets in the same UserWidget.  If you need to create links between widgets this is valuable.  Will likely introduce new ways to utilize this in the future, for now just getting it working.

Change 3634070 by Arciel.Rekman

	Display actually used values of material overrides.

Change 3634254 by Arciel.Rekman

	Fix ResavePackages working poorly with projects on other drives (UE-49465).

	#jira UE-49465

Change 3635985 by Matt.Kuhlenschmidt

	Fixed typo in function name used by maps

	PR #3975: Add tooltip to Arrays in Editor (Contributed by projectgheist)


Change 3636012 by Matt.Kuhlenschmidt

	PR #3982: Unhide mouse cursor after using Ansel (Contributed by projectgheist)


Change 3636706 by Lauren.Ridge

	Epic Friday: Save parameters to child or sibling  instance functionality

Change 3638706 by Jamie.Dale

	Added an improved Japanese font to the editor

	This is only used when displaying Japanese text when the editor is set to Japanese, and uses a font with Japanese-style unified Han characters (our default fallback font uses Chinese-style unified Han characters).

	#jira UE-33268

Change 3639438 by Arciel.Rekman

	Linux: Repaired ARM server build (UE-49635).

	- Made Steam* plugins compile.
	- Disabled OpenEXR as the libs aren't compiled (need to be done separately).

	(Edigrating CL 3639429 from Release-4.17 to Dev-Editor)

Change 3640625 by Matt.Kuhlenschmidt

	PR #4012: FSlateApplication::ProcessReply use &Reply (Contributed by projectgheist)


Change 3640626 by Matt.Kuhlenschmidt

	PR #4011: Remove space from filename (Contributed by projectgheist)


Change 3640697 by Matt.Kuhlenschmidt

	PR #4010: PNG alpha fix (Contributed by mmdanggg2)


Change 3641137 by Jamie.Dale

	Fixed an issue where a culture specific sub-font could produce incorrect measurements during a culture switch

	It would fallback to the last resort font for a frame or two while the font cache flushed. This has it update the ranges immediately.

Change 3641351 by Jamie.Dale

	Fixing incorrect weights on the Japanese sub-font

Change 3641356 by Jamie.Dale

	Fixing inconsistent font sizes between CoreStyle and EditorStyle

Change 3641710 by Jamie.Dale

	Fixed pure-virtual function call on UMulticastDelegateProperty

Change 3641941 by Lauren.Ridge

	Adding a Parameter Details tab to the Material Editor so users can change default parameter details

Change 3644141 by Jamie.Dale

	Added an improved Korean font to the editor

	This is only used when displaying Korean text when the editor is set to Korean

Change 3644213 by Arciel.Rekman

	Fix the side effects of a fix for UE-49465.

	- Default materials were apparently not being found while building DDC (e.g. making an installed build), now they are
	  and we should not reset loaders on them lest we trigger HasDefaultMaterialsPostLoaded() assert later.

	#jira UE-49465

Change 3644777 by Jamie.Dale

	Reverting Korean editor font back to NanumGothic as NanumBarunGothic looked too squished

Change 3644879 by tim.gautier

	QAGame: Optimized assets for Procedural Foliage testing
	- Added camera bookmarks to Stations in QA-Foliage
	- Renamed QA-FoliageTypeInst assets to ProcFoliage_Shape
	- Fixed up redirectors

Change 3645109 by Matt.Kuhlenschmidt

	PR #3990: Git plugin: fix status of renamed, removed, missing, untracked assets (Contributed by SRombauts)


Change 3645114 by Matt.Kuhlenschmidt

	PR #3991: Git Plugin: Fix RunDumpToFile() leaking Process handles (Contributed by SRombauts)


Change 3645116 by Matt.Kuhlenschmidt

	PR #3996: Git Plugin: run an "UpdateStatus" at "Connect" time to populate the Source Control cache (Contributed by SRombauts)


Change 3645118 by Matt.Kuhlenschmidt

	PR #4005: Git Plugin: Expand the size of the Button "Initialize project with Git" (Contributed by SRombauts)


Change 3645876 by Arciel.Rekman

	Linux: fix submenus of context menu not working (UE-47639).

	- Change by icculus (Ryan Gordon).
	- QA-ClickHUD seems to be not affected by this change (it is already broken alas).

	#jira UE-47639

Change 3648088 by Jamie.Dale

	Fixed some case-sensitivity issues with FText format argument names/pins

	These were originally case-sensitive, but that was lost somewhere along the way. This change restores their original behavior.

	#jira UE-47122

Change 3648097 by Jamie.Dale

	Moved common macOS/iOS localization implementation into FApplePlatformMisc

	#jira UE-49940

Change 3650858 by Arciel.Rekman

	UBT: improve CodeLite project generator (UE-49400).

	- PR #3987 submitted by yaakuro (Cengiz Terzibas).

	#jira UE-49400

Change 3651231 by Arciel.Rekman

	Linux: default to SM5 for Vulkan.

	- Change by Timothee.Bessett.

Change 3653627 by Matt.Kuhlenschmidt

	PR #4020: Source Control Submit Files now interprets Escape key as if the user clicked cancel (Contributed by SRombauts)


Change 3653628 by Matt.Kuhlenschmidt

	PR #4022: Add New C++ Class dialog remember previously selected module. (Contributed by Koderz)


Change 3653984 by Jamie.Dale

	Fixed some redundant string construction

Change 3658528 by Joe.Graf

	UE-45141 - Added CMAKE_CXX_COMPILER and CMAKE_C_COMPILER settings to the generated CMake files

Change 3658594 by Jamie.Dale

	Zipping in UAT now always uses UTF-8 encoding to prevent Unicode issues

	#jira UE-27263

Change 3659643 by Michael.Trepka

	Added a call to FCoreDelegates::ApplicationWillTerminateDelegate.Broadcast(); in Mac RequestExit() to match Windows behavior

	#jira UETOOL-1238

Change 3661908 by Matt.Kuhlenschmidt

	USD asset importing improvements

Change 3664100 by Matt.Kuhlenschmidt

	Fix static analysis

Change 3664107 by Matt.Kuhlenschmidt

	PR #4051: UE-49448: FPropertyChangedEvent to include TopLevelObjects (Contributed by projectgheist)


Change 3664125 by Matt.Kuhlenschmidt

	PR #4036: Add missing GRAPHEDITOR_API (Contributed by projectgheist)


Change 3664340 by Jamie.Dale

	PR #3648: Prevent GatherTextFromSource from failing the commandlet (Contributed by projectgheist)


Change 3664403 by Jamie.Dale

	PR #3769: Fixes UE-46973 - Drag and Dropping Folders with Names (Contributed by LordNed)


Change 3664539 by Jamie.Dale

	PR #3280: Added EditableText functionality (Contributed by projectgheist)


Change 3665433 by Alexis.Matte

	When we finish importing morph target we must re-initialise the render resources since we now use GPU morph target.
	#jira UE-50231

Change 3666747 by Cody.Albert



Change 3669280 by Jamie.Dale

	PR #4060: UE-50455: Verify folder is newly created before removing from tree (Contributed by projectgheist)


Change 3669718 by Jamie.Dale

	PR #4061: Clear Content Browser folder search box on escape key (Contributed by projectgheist)


Change 3670838 by Alexis.Matte

	Fix crash when deleting a skeletal mesh LOD and the mouse is over the "reimport" button.
	#jira UE-50387

Change 3671559 by Matt.Kuhlenschmidt

	Update SimpleUI automation test ground truth

	#jira UE-50325

Change 3671587 by Alexis.Matte

	Fix fbx importer scale not always apply. A cache array was not reset when opening a fbx file.
	#jira UE-50147

Change 3671730 by Jamie.Dale

	Added PostInitInstance to UClass to allow class types to perform construction time initialization of their instances

Change 3672104 by Michael.Dupuis

	#jira UE-50427: Update the volume visibility list of the editor viewport when changing the procedural foliage settings

Change 3674906 by Alexis.Matte

	Make sure the export LOD option is taken in consideration when exporting a level or the current level selection
	#jira UE-50248

Change 3674942 by Matt.Kuhlenschmidt

	Fix static analysis

Change 3675401 by Alexis.Matte

	-fix export animation, do not truncate the last frame anymore
	-fix the import animation, there was a display issue in the progress bar. Also a floorToInt sometime truncate the last valid frame. We also have a better way to calculate the time increment we use to sample the fbx curves.

	#jira UE-48231

Change 3675990 by Alexis.Matte

	Remove morph target when doing a re-import, so morph will be remove if they do not exist anymore in the fbx.
	This is to avoid driving random vertex with old morph target.
	#jira UE-50391

Change 3676169 by Alexis.Matte

	When we re-import with dialog the option, "Override Full Name" was set to false and save with the option dialog. We now not set it to false, since it was not use during re-import.

Change 3676396 by Alexis.Matte

	Make all LOD 0 name consistent in staticmesh editor
	#jira UE-49461

Change 3677730 by Cody.Albert

	Enable locking of Persistent Level in Levels tab

	#jira UE-50686

Change 3677838 by Jamie.Dale

	Replaced broken version of Roboto Light

Change 3679619 by Alexis.Matte

	Integrate GitHub pr #4029 to fix import fbx chunk material assignation.
	#jira UE-50001

Change 3680093 by Alexis.Matte

	Fix the skeletal mesh so the vertex color is part of the vertex equality like with the static mesh.

Change 3680931 by Arciel.Rekman

	SlateDialogs: show image icon for *.tga (UE-25106).

	- Also reworked the logic somewhat.

	#jira UE-25106

Change 3681966 by Yannick.Lange

	MaterialEditor post-process preview.
	#jira UE-45307

Change 3682407 by Lauren.Ridge

	Fixes for material editor compile errors

Change 3682628 by Lauren.Ridge

	Content browser filters for Material Layers, Blends, and their instances

Change 3682725 by Lauren.Ridge

	Adding filter assets and instance assets to Material Layers and Material Layer Blends. Turning Material Layering on by default

Change 3682921 by Lauren.Ridge

	Fix for instance layers not initializing fully

Change 3682954 by Lauren.Ridge

	Creating Material Layer Test Assets

Change 3683582 by Alexis.Matte

	Fix static analysis build

Change 3683614 by Matt.Kuhlenschmidt

	PR #4062: Git Plugin: Fix UE-44637: Deleting an asset is unsuccessful if the asset is marked for add (Contributed by SRombauts)


Change 3684130 by Lauren.Ridge

	Allow visible parameter retrieval to correctly recurse through internally called functions. Previous check was intended to prevent function previews from leaving their graph through unhooked inputs, but unintentionally blocked all function inputs.

Change 3686289 by Arciel.Rekman

	Remove the pessimization (UE-23791).

Change 3686455 by Lauren.Ridge

	Fixes for adding/removing a layer parameter from the parent not updating the child

Change 3686829 by Jamie.Dale

	No longer include trailing whitespace in the justification calculation for soft-wrapped lines

	#jira UE-50266

Change 3686970 by Lauren.Ridge

	Making material parameter preview work for functions as well

Change 3687077 by Jamie.Dale

	Fixed crash using FActorDetails with the struct details panel

Change 3687152 by Jamie.Dale

	Fixed the row structure tag not appearing in the Content Browser for Data Table assets

	The CDO is used to filter these tags, and the CDO was omiting that tag which caused it to be filtered for all Data Tables.

	#jira UE-48691

Change 3687174 by Lauren.Ridge

	Fix for material layer sub-parameters showing up in the default material parameters panel

Change 3688100 by Lauren.Ridge

	Fixing static analysis error

Change 3688317 by Jamie.Dale

	Fixed crash using the widget reflector in a cooked game

	Editor-style isn't available in cooked games. Core-style should be used instead for the widget reflector.

Change 3689054 by Jamie.Dale

	Reference Viewer can now show/copy references lists for nodes with multiple objects, or multiple selected nodes

	#jira UE-45751

Change 3689513 by Jamie.Dale

	Fixed justification bug with RTL text caused by CL# 3686829

	Also implemented the same alignment fix for visually left-aligned RTL text.

	#jira UE-50266

Change 3690231 by Lauren.Ridge

	Added Material Layers Parameters Preview (all editing disabled) panel to the Material Editor

Change 3690234 by Lauren.Ridge

	Adding Material Layers Function Parameter to Static Parameter Compare

Change 3690750 by Chris.Bunner

	Potential nullptr crash.

Change 3690751 by Chris.Bunner

	Fixed logic on overridden vector parameter retrieval for material instances checking a function owned parameter.

Change 3691010 by Jamie.Dale

	Fixed some clipping issues that could occur with right-aligned text

	FTextBlockLayout::OnPaint was passing an unscaled offset to SetVisibleRegion, and it also wasn't correctly adjusting the offset for RTL text with left-alignment (which becomes a visual right-alignment)

	#jira UE-46760

Change 3691091 by Jamie.Dale

	Renamed FTextBlockLayout to FSlateTextBlockLayout to reflect that it's a Slate specific type

Change 3691134 by Alexis.Matte

	Make sure we instance also the collision mesh when exporting a level to fbx file.
	#jira UE-51066

Change 3691157 by Lauren.Ridge

	Fix for reset to default not refreshing sub-parameters

Change 3691192 by Jamie.Dale

	Fixed Content Browser selection resetting when changing certain view settings

	#jira UE-49611

Change 3691204 by Alexis.Matte

	Remove fbx export file version 2010 compatibility. The 2018 fbx sdk refuse to export earlier then 2011.
	#jira UE-51023

Change 3692335 by Lauren.Ridge

	Setting displayed asset to equal filter asset if no instance has been selected

Change 3692479 by Jamie.Dale

	Fixed whitespace

Change 3692508 by Alexis.Matte

	Make sure we warn the user that there is nothing to export when exporting to fbx using "export selected" or "export All" from the file menu.
	We also prevent the export dialog to show
	#jira UE-50973

Change 3692639 by Jamie.Dale

	Translation Editor now shows stale translations as "Untranslated"

Change 3692743 by Lauren.Ridge

	Smaller blend icons, added icon size override to FObjectEntryBox

Change 3692830 by Alexis.Matte

	Fix linux build

Change 3692894 by Lauren.Ridge

	Tooltip on "Parent" in material layers

Change 3693141 by Jamie.Dale

	Removed dead code

	FastDecimalFormat made this redundant

Change 3693580 by Jamie.Dale

	Added AlwaysSign number formatting option

	#jira UE-10310

Change 3693784 by Jamie.Dale

	Fixed assert extracting the number formatting rules for Arabic

	It uses a character outside the BMP for its plus and minus sign, so we need these to be a string to handle that.

	#jira UE-10310

Change 3694428 by Arciel.Rekman

	Linux: make directory watch request a warning so they don't block cooking.

	- See https://answers.unrealengine.com/questions/715206/cook-error-on-linux.html

Change 3694458 by Matt.Kuhlenschmidt

	Made duplicate keybinding warning non-fatal

Change 3694496 by Alexis.Matte

	fix static analysis build

Change 3694515 by Jamie.Dale

	Added support for culture correct parsing of decimal numbers

	#jira UE-4028

Change 3694621 by Jamie.Dale

	Added a variant of FastDecimalFormat::StringToNumber that takes a string length

	This can be useful if you want to convert a number from within a non-null terminated string

	#jira UE-4028

Change 3694958 by Jamie.Dale

	Added a parsed length output to FastDecimalFormat::StringToNumber to allow permissive parsing

	You can test this rather than the result if you want to attempt to parse a number from a string that may have other data after it. This also fixes the sign-suffix causing the parsing to fail.

	#jira UE-4028

Change 3695083 by Alexis.Matte

	Optimisation of the morph target import
	- We now compute only the normal for the shape the tangent are not necessary
	- The async tasks are create when there is some available cpu thread to avoid filling the memory
	- When we re-import the morph target are deleted in bulk avoiding to initialize the morph map for every morphs targets
	#jira UE-50945

Change 3695122 by Jamie.Dale

	GetCultureAgnosticFormattingRules no longer returns a copy

Change 3695835 by Arciel.Rekman

	TestPAL: greatly expanded malloc test.

Change 3695918 by Arciel.Rekman

	TestPAL: Added thread priority test.

Change 3696589 by Arciel.Rekman

	TestPAL: tweak thread priorities test (better readability).

Change 3697345 by Alexis.Matte

	Fix reorder of material when importing a LOD with new material
	#jira UE-51135

Change 3699590 by Jamie.Dale

	Updated SGraphPinNum to use a numeric editor

	#jira UE-4028

Change 3699698 by Matt.Kuhlenschmidt

	Fix crash opening the level viewport context menu if the actor-component selection is out of sync

	#jira UE-48444

Change 3700158 by Arciel.Rekman

	Enable packaging for Android Vulkan on Linux (UETOOL-1232).

	- Change by Cengiz Terzibas

Change 3700224 by Arciel.Rekman

	TestPAL: fixed a memory leak.

Change 3700775 by Cody.Albert

	Don't need to initialize EnvironmentCubeMap twice.

Change 3700866 by Michael.Trepka

	PR #3223: Remove unnecessary reallocation. (Contributed by foollbar)


	#jira UE-41643

Change 3701132 by Michael.Trepka

	Copy of CL 3671538

	Fixed issues with editor's game mode in high DPI on Mac.

	#jira UE-49947, UE-51063

Change 3701421 by Michael.Trepka

	Fixed a crash in FScreenShotManager caused by an attempt to access a deleted FString in async lambda expression

Change 3701495 by Alexis.Matte

	Fix fbx importer "import normals" option when mix with "mikkt" tangent build it was recomputing the normals instead of importing them.
	#jira UE-UE-51359

Change 3702982 by Jamie.Dale

	Cleaned up some localization setting names

	These now have consistent names and avoid double negatives. This also fixes needing to restart the editor when changing the "ShouldUseLocalizedPropertyNames" setting.

Change 3703517 by Arciel.Rekman

	TestPAL: improved thread test.

	- Changed the counter to a normal variable to reduce possible contentions (threads used to share the counter in an early prototype, hence the usage of an atomic).

Change 3704378 by Michael.Trepka

	Disable Zoom button on Mac if project requests a resizeable window without it.

	#jira UE-51335

Change 3706316 by Jamie.Dale

	Fixed the asset search suggestions list closing if you clicked on its scrollbar

	#jira UE-28885

Change 3706855 by Alexis.Matte

	Support importing animation that has some keys with negative time
	#jira UE-51305

Change 3709634 by Matt.Kuhlenschmidt

	PR #4146: Null access check on ForceLOD in FViewport::HighResScreenshot (Contributed by projectgheist)


Change 3711085 by Michael.Trepka

	Reenabled UBT makefiles on Mac

Change 3713049 by Josh.Engebretson

	The ConfigPropertyEditor now generates a unique runtime UClass.  It uses the outer name on the property instead of a unique ID as a unique id would generate a new UClass every time (and these are RF_Standalone).  I also removed some static qualifiers for Section and Property names which were incorrect.

	#jira UE-51319

Change 3713144 by Lauren.Ridge

	Fixing automated test error

	#jira UE-50982

Change 3713395 by Alexis.Matte

	Fix auto import mountpoint
	#jira UE-51524

Change 3713881 by Michael.Trepka

	Added -buildscw to Mac Build.sh script to build ShaderCompileWorker in addition to the requested target. Xcode passes it to the script when building non-program targets.

	#jira UE-31093

Change 3714197 by Michael.Trepka

	Send IMM key down event to the main window instead of Cocoa key window, as that's what the Slate's active window is. This solves problems with IMM not working in context menu text edit fields.

	#jira UE-47915

Change 3714911 by Joe.Graf

	Merge of cmake changes from Dev-Rendering

Change 3715973 by Michael.Trepka

	Disable OS close button on Windows if project settings request that

	#jira UE-45522

Change 3716390 by Lauren.Ridge

	The color picker summoned when double-clicking vector3 nodes now has its intended "do not refresh until OK is clicked" behavior.

	#jira UE-50916

Change 3716529 by Josh.Engebretson

	Content Browser: Clamp "Assets to Load at Once Before Warning" so it cannot be set below 1

	#jira UE-51341

Change 3716885 by Josh.Engebretson

	Tracking transactions such as a duplication operation can modify a selection which differs from the initial one.  Added package state tracking to restore unmodified state when necessary.

	#jira UE-48572

Change 3716929 by Josh.Engebretson

	Unshelved from pending changelist '3364093':

	PR #3420: Exe's icons and properties (Contributed by projectgheist)


Change 3716937 by Josh.Engebretson

	Unshelved from pending changelist '3647428':

	PR #4026: Fixed memory leaks for pipe writes and added data pipe writes (Contributed by Hemofektik)


Change 3717002 by Josh.Engebretson

	Fix FileReference/string conversion

Change 3717355 by Joe.Graf

	Fixed CMake file generation on Windows including Engine/Source/ThirdParty source

Change 3718256 by Arciel.Rekman

	TestPAL: slight mod to the malloc test.

	- Touch the allocated memory to check actual resident usage.

Change 3718290 by Arciel.Rekman

	BAFO: place descriptor after the allocation to save some VIRT memory.

	- We're relying on passing correct "Size" argument to Free() anyway, and this modification makes use of that extra information to save on memory for the descriptor.

Change 3718508 by Michael.Trepka

	Fixed vsnprintf on platforms that use our custom implementation in StandardPlatformString.cpp to ignore length modifier for certain types (floating point, pointer)

	#jira UE-46148

Change 3718855 by Lauren.Ridge

	Adding content browser favorite folders. Add or remove folders from the favorite list in the folder's right-click context menu, and hide or show the favorites list in the Content Browser options.

Change 3718932 by Cody.Albert

	Update ActorSequence plugin loading phase to PreDefault

	#jira UE-51612

Change 3719378 by tim.gautier

	QAGame: Renamed multiTxt_Justification > UMG_TextJustification.
	Added additional Text Widgets for testing

Change 3719413 by Lauren.Ridge

	Resubmit of content browser favorites

Change 3719803 by Yannick.Lange

	VREditor: Fix crash with null GEditor
	#jira UE-50103

Change 3721127 by tim.gautier

	QAGame: Fixed up a ton of redirectors within /Content and /Content/Materials
	- Added M_ParamDefaults and MF_ParamDefaults
	- Moved legacy MeshPaint materials into /Content/Materials/MeshPaint
	- Renamed ColorPulse assets from MatFunction_ > MF_, moved into /Content/Materials/Functions

Change 3721255 by Alexis.Matte

	Replace skeletal mesh import option "keep overlapping vertex" by 3 float thresholds allowing the user to control the welding thresholds.
	#jira UE-51363

Change 3721594 by Lauren.Ridge

	Material Blends now have plane mesh previews in their icons.

Change 3722072 by tim.gautier

	QAGame: Updated MF_ParamDefaults - using red channel as roughness
	Updated M_ParamDefaults - tweaked Scalar values

Change 3722180 by Michael.Trepka

	Updated Xcode project generator to sort projects in the navigator by name (within folders) and also sort the list of schemes so that their order matches the order of projects in the navigator.

	#jira UE-25941

Change 3722220 by Michael.Trepka

	Fixed a problem with Xcode project generator not handling quoted preprocessor definitions correctly

	#jira UE-40246

Change 3722806 by Lauren.Ridge

	Fixing non-editor compiles

Change 3722914 by Alexis.Matte

	Fbx importer: Add new attribute type(eSkeleton) for staticmesh socket import.

	#jira UE-51665

Change 3723446 by Michael.Trepka

	Copy of CL 3688862 from 4.18 + one more fix for a deadlock related to window resizing when using IME

	Don't do anything in Mac window's windowWillResize: if we're simply chaning the z order of windows. This way we avoid a rare dead lock when hiding the window.

	#jira UE-48257

Change 3723505 by Matt.Kuhlenschmidt

	Fix duplicate actors being created for USD primitives that specify a custom actor class

Change 3723555 by Matt.Kuhlenschmidt

	Fix crash loading the gameplayabilities module

	#jira UE-51693

Change 3723557 by Matt.Kuhlenschmidt

	Fixed tooltip on viewport dpi scaling option

Change 3723870 by Lauren.Ridge

	Fixing incorrect reset to default visibility, adding clear behavior to fields

Change 3723917 by Arciel.Rekman

	Linux: fix compilation with glibc 2.26+ (UE-51699).

	- Fixes compilation on Ubuntu 17.10 among others.

	(Merging 3723489 from //UE4/Release-4.18/... to //UE4/Dev-Editor/...)

Change 3723918 by Arciel.Rekman

	Linux: do not test for popcnt presence unnecessarily (UE-51677).

	(Merging 3723904 from //UE4/Release-4.18/... to //UE4/Dev-Editor/...)

Change 3724229 by Arciel.Rekman

	Fix FOutputDeviceStdOutput to use printf() on Unix platforms.

Change 3724261 by Arciel.Rekman

	TestPAL: fix thread priority test (zero the counter).

Change 3724978 by Arciel.Rekman

	Linux: fix priority calculation.

	- Rlimit values are always positive, so this was completely broken when the RLIMIT_NICE is non-0.

Change 3725382 by Matt.Kuhlenschmidt

	Guard against crashes and add more logging when actor creation fails.
	Looks like it could be manual garbage collections triggered before conversion is complete so those have been removed

	#jira UE-47464

Change 3725559 by Matt.Kuhlenschmidt

	Added a setting to enable/disable high dpi support in editor.   This currently only functions in Windows.
	Moved some files around for better consistency

Change 3725640 by Arciel.Rekman

	Fix Linux thread/process priorities.

	- Should also speed up SCW on Linux by deprioritizing them less.

Change 3726101 by Matt.Kuhlenschmidt

	Fix logic bug in USD child "kind" type resolving

Change 3726244 by Joe.Graf

	Added an option to generate a minimal set of targets for cmake files
	Added shader and config files to cmake file generation for searching within IDEs

Change 3726506 by Arciel.Rekman

	Fix compile issue after DPI change.

Change 3726549 by Matt.Kuhlenschmidt

	Remove unnecessary indirection to cached widgets in the hit test grid

Change 3726660 by Arciel.Rekman

	Enable DPI switch on Linux.

Change 3726763 by Arciel.Rekman

	Fix mismatching "noperspective" qualifier (UE-50807).

	- Pull request #4080 by TTimo.

Change 3727080 by Michael.Trepka

	Added support for editor's EnableHighDPIAwareness setting on Mac

Change 3727658 by Matt.Kuhlenschmidt

	Fix shutdown crash if level editor is still referenced after the object system has been gc'd

	#jira UE-51630

Change 3728270 by Matt.Kuhlenschmidt

	Remove propertyeditor dependency from editorstyle

Change 3728291 by Arciel.Rekman

	Linux: fix for a crash on a headless system (UE-51714).

	- Preliminary change before merging to 4.18.

Change 3728293 by Arciel.Rekman

	Linux: remove unneeded dependency on CEF.

	- Old workaround should no longer be needed, while this dependency makes UE4 depend on a ton of external libs.

Change 3728524 by Michael.Trepka

	Copy of CL 3725570

	Removed Enable Fullscreen option from editor's Window menu on Mac. Windowed fullscreen mode is currently unavailable on Mac in editor mode as supporting it properly would require it to work with multiple spaces and split screen, which we currently don't handle (requested in UE-27240)

	#jira UE-51709

Change 3728875 by Michael.Trepka

	Fixed compile error in Mac SlateOpenGLContext.cpp

Change 3728880 by Matt.Kuhlenschmidt

	Guard against invalid worlds in thumbnail renderers

Change 3728924 by Michael.Trepka

	Don't defer MacApplication->CloseWindow() call. This should fix a rare problem with deferred call executing during Slate's PrepassWindowAndChildren call.

	#jira UE-51711

Change 3729288 by Joe.Graf

	Added the .idea/misc.xml file generation to speed up CLion indexing

Change 3729935 by Michael.Dupuis

	#jira UE-51722: Hide from UI invalid enum values

Change 3730234 by Matt.Kuhlenschmidt

	Fix "Game Gets Mouse Control" setting no longer functioning and instead the mouse was always captured.

	#jira UE-51801

Change 3730349 by Michael.Dupuis

	#jira UE-51324: Clear the UI selection when rebuilding the palette, as we destroyed all items and recreate them, so selection is on invalid item

Change 3730438 by Lauren.Ridge

	Cleaning up material layering UI functions

Change 3730723 by Jamie.Dale

	Fixed FastDecimalFormat::StringToNumber incorrectly reporting that number-like sequences that lacked digits had been parsed as numbers

	#jira UE-51799

Change 3731008 by Lauren.Ridge

	Changing Layers and Blends from proxy assets to real assets

Change 3731026 by Arciel.Rekman

	libelf: make elf_end() visible (UE-51843).

	- This repairs compilation for a case when CUDA is being used.
	- Also added some missing files for ARM 32-bit.

Change 3731081 by Lauren.Ridge

	New material layer test assets

Change 3731186 by Josh.Engebretson

	Adding camera speed scalar setting and Toolbar UI to increase range on camera speed presets

	#jira UE-50104

Change 3731188 by Mike.Erwin

	Improve responsiveness of Open Asset dialog.

	On large projects, there's a noticeable delay when opening and searching/filtering assets.

	Stopwatch measurements on my machine (seconds for ~122,000 assets):
		before	with this CL
	ctrl-P	1.4	0.45
	search	1.8	0.55

	CollectionManagerModule was the main culprit for search/filter slowness.

	Open Asset delay was due to filtering out plugin content. We were doing a lot of redundant work for what is essentially a read-only operation.

Change 3731682 by Arciel.Rekman

	UnrealEd: Allow unattended commandlets to rename/save packages.

Change 3732305 by Michael.Dupuis

	#jira UE-48434 : Only register if the foliage type still has a valid mesh

Change 3732361 by Matt.Kuhlenschmidt

	Fix two settings objects being created in the transient package with the same name

	#jira UE-51891

Change 3732895 by Josh.Engebretson

	https://jira.it.epicgames.net/browse/UE-51706

	If a shared DDC is not being used, present a notification to the licensee with a link on how to setup a shared DDC.
	Adds DDC notification events for check/put and query for whether a shared DDC is in use.

	#jira UE-51706

Change 3733025 by Arciel.Rekman

	UBT: make sure new clang versions are invoked.

Change 3733311 by Mike.Erwin

	Fix Linux compile warning from CL 3731188

	It didn't like mixing && and || without parentheses. Reworked logic to do one test at a time, put cheaper tests first to avoid calls to more expensive IsPluginFolder.

Change 3733658 by Josh.Engebretson

	Add a missing #undef LOCTEXT_NAMESPACE

Change 3734003 by Arciel.Rekman

	Fix Windows attempting to use printf %ls and crashing at that (UE-51934).

Change 3734039 by Michael.Trepka

	Fixed a couple of merge issues in Mac ApplicationCore

Change 3734052 by Michael.Trepka

	One more Mac ApplicationCore fix

Change 3734244 by Lauren.Ridge

	Fix for accessing Slate window on render thread

Change 3734950 by Josh.Engebretson

	Fixing clang warning

Change 3734978 by Jamie.Dale

	Relaxed enum property importing to allow valid numeric values to be imported too

	This was previously made more strict which caused a regression in Data Table importing

	#jira UE-51848

Change 3734999 by Arciel.Rekman

	Linux: add LTO support and more.

	- Adds ability to use link-time opitimization (reusing current target property bAllowLTCG).
	- Supports using llvm-ar and lld instead of ar/ranlib and ld.
	- More build information printed (and in a better organized way).
	- Native scripts updated to install packages with the appropriate tools on supported systems
	- AutoSDKs updated to require a new toolchain (already checked in).
	- Required disabling OpenAL due to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219089

Change 3735268 by Matt.Kuhlenschmidt

	Added support for canvas based DPI scaling.

	-Scene canvas is by default not scaled as this could severely impact any game using a canvas based UI
	-The debug canvas for stats is always dpi scaled in editor and pie.
	-Eliminated text scaling workaround now that the entire canvas is properly scaled
	-Enabled canvas scaling in cascade UI

Change 3735329 by Matt.Kuhlenschmidt

	Fix potential crash if an asset editor has an object deleted out from under it

	#jira UE-51941



Change 3735502 by Arciel.Rekman

	Fix compile issue (bShouldUpdateScreenPercentage).

Change 3735878 by Jamie.Dale

	Updated FString::SanitizeFloat to allow you to specify the min number of fractional digits to have in the resultant string

	This defaults to 1 as that was the old behavior of FString::SanitizeFloat, but can also be set to 0 to prevent adding .0 to whole numbers.

Change 3735881 by Jamie.Dale

	JsonValue no longer stringifies whole numbers as floats

Change 3735884 by Jamie.Dale

	Only allow enums to import integral values

Change 3735912 by Josh.Engebretson

	Improving cook process error/warning handling including asset warning/error content browser links and manual dismiss for cook error notifications

	#jira UE-48131

Change 3736280 by Matt.Kuhlenschmidt

	Fix 0 dpi scale for canvases

	#jira UE-51995

Change 3736298 by Matt.Kuhlenschmidt

	Force focus of game viewports in vr mode

Change 3736374 by Jamie.Dale

	Fixed some places where input chords were being used without testing that they had a valid key set

	#jira UE-51799

Change 3738543 by Matt.Kuhlenschmidt

	Better fix for edit condition crashes

	#jira UE-51886

Change 3738603 by Lauren.Ridge

	Copy over of drag and drop non-array onto array fix

Change 3739701 by Chris.Babcock

	Fix crashlytics merge error
	#jira UE-52064
	#ue4
	#android

[CL 3739980 by Matt Kuhlenschmidt in Main branch]
2017-11-06 18:22:01 -05:00

2140 lines
72 KiB
C++

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.
#include "CoreMinimal.h"
#include "HAL/PlatformProcess.h"
#include "GenericPlatform/GenericPlatformFile.h"
#include "HAL/FileManager.h"
#include "Misc/CoreMisc.h"
#include "Misc/Paths.h"
#include "Misc/QueuedThreadPool.h"
#include "Misc/OutputDeviceNull.h"
#include "Stats/Stats.h"
#include "Async/AsyncWork.h"
#include "Containers/Ticker.h"
#include "Misc/ConfigCacheIni.h"
#include "Misc/FeedbackContext.h"
#include "Misc/ScopedSlowTask.h"
#include "Misc/App.h"
#include "Modules/ModuleManager.h"
#include "UObject/ObjectMacros.h"
#include "UObject/UObjectGlobals.h"
#include "Serialization/ArchiveUObject.h"
#include "UObject/GarbageCollection.h"
#include "UObject/Class.h"
#include "UObject/UObjectIterator.h"
#include "UObject/UnrealType.h"
#include "Misc/PackageName.h"
#include "IHotReload.h"
#include "IDirectoryWatcher.h"
#include "DirectoryWatcherModule.h"
#include "HotReloadLog.h"
#include "AnalyticsEventAttribute.h"
#include "Interfaces/IAnalyticsProvider.h"
#include "ProfilingDebugging/ScopedTimers.h"
#include "IPluginManager.h"
#include "DesktopPlatformModule.h"
#if WITH_ENGINE
#include "Engine/Engine.h"
#include "Kismet2/KismetReinstanceUtilities.h"
#include "HotReloadClassReinstancer.h"
#include "EngineAnalytics.h"
#endif
#include "Misc/ScopeExit.h"
#if WITH_EDITOR
#include "Editor.h"
#endif
DEFINE_LOG_CATEGORY(LogHotReload);
#define LOCTEXT_NAMESPACE "HotReload"
namespace EThreeStateBool
{
enum Type
{
False,
True,
Unknown
};
static bool ToBool(EThreeStateBool::Type Value)
{
switch (Value)
{
case EThreeStateBool::False:
return false;
case EThreeStateBool::True:
return true;
default:
UE_LOG(LogHotReload, Fatal, TEXT("Can't convert EThreeStateBool to bool value because it's Unknown"));
break;
}
return false;
}
static EThreeStateBool::Type FromBool(bool Value)
{
return Value ? EThreeStateBool::True : EThreeStateBool::False;
}
};
/**
* Module for HotReload support
*/
class FHotReloadModule : public IHotReloadModule, FSelfRegisteringExec
{
public:
FHotReloadModule()
{
ModuleCompileReadPipe = nullptr;
bRequestCancelCompilation = false;
bIsAnyGameModuleLoaded = EThreeStateBool::Unknown;
bDirectoryWatcherInitialized = false;
}
/** IModuleInterface implementation */
virtual void StartupModule() override;
virtual void ShutdownModule() override;
/** FSelfRegisteringExec implementation */
virtual bool Exec( UWorld* Inworld, const TCHAR* Cmd, FOutputDevice& Ar ) override;
/** IHotReloadInterface implementation */
virtual void Tick() override;
virtual void SaveConfig() override;
virtual bool RecompileModule(const FName InModuleName, const bool bReloadAfterRecompile, FOutputDevice &Ar, bool bFailIfGeneratedCodeChanges = true, bool bForceCodeProject = false) override;
virtual bool IsCurrentlyCompiling() const override { return ModuleCompileProcessHandle.IsValid(); }
virtual void RequestStopCompilation() override { bRequestCancelCompilation = true; }
virtual void AddHotReloadFunctionRemap(FNativeFuncPtr NewFunctionPointer, FNativeFuncPtr OldFunctionPointer) override;
virtual ECompilationResult::Type RebindPackages(TArray< UPackage* > Packages, TArray< FName > DependentModules, const bool bWaitForCompletion, FOutputDevice &Ar) override;
virtual ECompilationResult::Type DoHotReloadFromEditor(const bool bWaitForCompletion) override;
virtual FHotReloadEvent& OnHotReload() override { return HotReloadEvent; }
virtual FModuleCompilerStartedEvent& OnModuleCompilerStarted() override { return ModuleCompilerStartedEvent; }
virtual FModuleCompilerFinishedEvent& OnModuleCompilerFinished() override { return ModuleCompilerFinishedEvent; }
virtual FString GetModuleCompileMethod(FName InModuleName) override;
virtual bool IsAnyGameModuleLoaded() override;
private:
/**
* Enumerates compilation methods for modules.
*/
enum class EModuleCompileMethod
{
Runtime,
External,
Unknown
};
/**
* Helper structure to hold on to module state while asynchronously recompiling DLLs
*/
struct FModuleToRecompile
{
/** Name of the module */
FString ModuleName;
/** Desired module file name suffix, or empty string if not needed */
FString ModuleFileSuffix;
/** The module file name to use after a compilation succeeds, or an empty string if not changing */
FString NewModuleFilename;
};
/**
* Helper structure to store the compile time and method for a module
*/
struct FModuleCompilationData
{
/** Has a timestamp been set for the .dll file */
bool bHasFileTimeStamp;
/** Last known timestamp for the .dll file */
FDateTime FileTimeStamp;
/** Last known compilation method of the .dll file */
EModuleCompileMethod CompileMethod;
FModuleCompilationData()
: bHasFileTimeStamp(false)
, CompileMethod(EModuleCompileMethod::Unknown)
{ }
};
/**
* Adds a callback to directory watcher for the game binaries folder.
*/
void RefreshHotReloadWatcher();
/**
* Adds a directory watch on the binaries directory under the given folder.
*/
void AddHotReloadDirectory(IDirectoryWatcher* DirectoryWatcher, const FString& BaseDir);
/**
* Removes a directory watcher callback
*/
void ShutdownHotReloadWatcher();
/**
* Performs hot-reload from IDE (when game DLLs change)
*/
void DoHotReloadFromIDE();
/**
* Performs internal module recompilation
*/
ECompilationResult::Type RebindPackagesInternal(TArray<UPackage*>&& Packages, TArray<FName>&& DependentModules, const bool bWaitForCompletion, FOutputDevice& Ar);
/**
* Does the actual hot-reload, unloads old modules, loads new ones
*/
ECompilationResult::Type DoHotReloadInternal(const TMap<FString, FString>& ChangedModuleNames, const TArray<UPackage*>& Packages, const TArray<FName>& InDependentModules, FOutputDevice& HotReloadAr);
/**
* Finds all references to old CDOs and replaces them with the new ones.
* Skipping UBlueprintGeneratedClass::OverridenArchetypeForCDO as it's the
* only one needed.
*/
void ReplaceReferencesToReconstructedCDOs();
#if WITH_ENGINE
void RegisterForReinstancing(UClass* OldClass, UClass* NewClass);
void ReinstanceClasses();
/**
* Called from CoreUObject to re-instance hot-reloaded classes
*/
void ReinstanceClass(UClass* OldClass, UClass* NewClass, const TMap<UClass*, UClass*>& OldToNewClassesMap);
#endif
/**
* Tick function for FTicker: checks for re-loaded modules and does hot-reload from IDE
*/
bool Tick(float DeltaTime);
/**
* Directory watcher callback
*/
void OnHotReloadBinariesChanged(const TArray<struct FFileChangeData>& FileChanges);
/**
* Strips hot-reload suffix from module filename.
*/
static void StripModuleSuffixFromFilename(FString& InOutModuleFilename, const FString& ModuleName);
/**
* Sends analytics event about the re-load
*/
static void RecordAnalyticsEvent(const TCHAR* ReloadFrom, ECompilationResult::Type Result, double Duration, int32 PackageCount, int32 DependentModulesCount);
/**
* Declares a function type that is executed after a module recompile has finished.
*
* ChangedModules: A map between the names of the modules that have changed and their filenames.
* bRecompileFinished: Signals whether compilation has finished.
* CompilationResult: Shows whether compilation was successful or not.
*/
typedef TFunction<void(const TMap<FString, FString>& ChangedModules, bool bRecompileFinished, ECompilationResult::Type CompilationResult)> FRecompileModulesCallback;
/**
* Tries to recompile the specified modules in the background. When recompiling finishes, the specified callback
* delegate will be triggered, passing along a bool that tells you whether the compile action succeeded. This
* function never tries to unload modules or to reload the modules after they finish compiling. You should do
* that yourself in the recompile completion callback!
*
* @param ModuleNames Names of the modules to recompile
* @param RecompileModulesCallback Callback function to execute after compilation finishes (whether successful or not.)
* @param bWaitForCompletion True if the function should not return until recompilation attempt has finished and callbacks have fired
* @param Ar Output device for logging compilation status
* @return True if the recompile action was kicked off successfully. If this returns false, then the recompile callback will never fire. In the case where bWaitForCompletion=false, this will also return false if the compilation failed for any reason.
*/
bool RecompileModulesAsync( const TArray< FName > ModuleNames, FRecompileModulesCallback&& InRecompileModulesCallback, const bool bWaitForCompletion, FOutputDevice &Ar );
/** Called for successfully re-complied module */
void OnModuleCompileSucceeded(FName ModuleName, const FString& NewModuleFilename);
/**
* Tries to recompile the specified DLL using UBT. Does not interact with modules. This is a low level routine.
*
* @param ModuleNames List of modules to recompile, including the module name and optional file suffix.
* @param Ar Output device for logging compilation status.
* @param bForceCodeProject Even if it's a non-code project, treat it as code-based project
*/
bool RecompileModuleDLLs(const TArray< FModuleToRecompile >& ModuleNames, FOutputDevice& Ar, bool bFailIfGeneratedCodeChanges, bool bForceCodeProject);
/** Returns arguments to pass to UnrealBuildTool when compiling modules */
static FString MakeUBTArgumentsForModuleCompiling();
/**
* Starts compiling DLL files for one or more modules.
*
* @param GameName The name of the game.
* @param ModuleNames The list of modules to compile.
* @param InRecompileModulesCallback Callback function to make when module recompiles.
* @param Ar
* @param bInFailIfGeneratedCodeChanges If true, fail the compilation if generated headers change.
* @param InAdditionalCmdLineArgs Additional arguments to pass to UBT.
* @param bForceCodeProject Compile as code-based project even if there's no game modules loaded
* @return true if successful, false otherwise.
*/
bool StartCompilingModuleDLLs(const FString& GameName, const TArray< FModuleToRecompile >& ModuleNames,
FRecompileModulesCallback&& InRecompileModulesCallback, FOutputDevice& Ar, bool bInFailIfGeneratedCodeChanges,
const FString& InAdditionalCmdLineArgs, bool bForceCodeProject);
/** Launches UnrealBuildTool with the specified command line parameters */
bool InvokeUnrealBuildToolForCompile(const FString& InCmdLineParams, FOutputDevice &Ar);
/** Checks to see if a pending compilation action has completed and optionally waits for it to finish. If completed, fires any appropriate callbacks and reports status provided bFireEvents is true. */
void CheckForFinishedModuleDLLCompile(const bool bWaitForCompletion, bool& bCompileStillInProgress, bool& bCompileSucceeded, FOutputDevice& Ar, bool bFireEvents = true);
/** Called when the compile data for a module need to be update in memory and written to config */
void UpdateModuleCompileData(FName ModuleName);
/** Called when a new module is added to the manager to get the saved compile data from config */
static void ReadModuleCompilationInfoFromConfig(FName ModuleName, FModuleCompilationData& CompileData);
/** Saves the module's compile data to config */
static void WriteModuleCompilationInfoToConfig(FName ModuleName, const FModuleCompilationData& CompileData);
/** Access the module's file and read the timestamp from the file system. Returns true if the timestamp was read successfully. */
bool GetModuleFileTimeStamp(FName ModuleName, FDateTime& OutFileTimeStamp) const;
/** Checks if the specified array of modules to recompile contains only game modules */
bool ContainsOnlyGameModules(const TArray< FModuleToRecompile >& ModuleNames) const;
/** Callback registered with ModuleManager to know if any new modules have been loaded */
void ModulesChangedCallback(FName ModuleName, EModuleChangeReason ReasonForChange);
/** Callback registered with PluginManager to know if any new plugins have been created */
void PluginMountedCallback(IPlugin& Plugin);
/** FTicker delegate (hot-reload from IDE) */
FTickerDelegate TickerDelegate;
/** Handle to the registered TickerDelegate */
FDelegateHandle TickerDelegateHandle;
/** Handle to the registered delegate above */
TMap<FString, FDelegateHandle> BinariesFolderChangedDelegateHandles;
/** True if currently hot-reloading from editor (suppresses hot-reload from IDE) */
bool bIsHotReloadingFromEditor;
/** New module DLLs */
TMap<FString, FString> NewModules;
/** Moduels that have been recently recompiled from the editor **/
TSet<FString> ModulesRecentlyCompiledInTheEditor;
/** Delegate broadcast when a module has been hot-reloaded */
FHotReloadEvent HotReloadEvent;
/** Array of modules that we're currently recompiling */
TArray< FModuleToRecompile > ModulesBeingCompiled;
/** Array of modules that we're going to recompile */
TArray< FModuleToRecompile > ModulesThatWereBeingRecompiled;
/** Last known compilation data for each module */
TMap<FName, TSharedRef<FModuleCompilationData>> ModuleCompileData;
/** Multicast delegate which will broadcast a notification when the compiler starts */
FModuleCompilerStartedEvent ModuleCompilerStartedEvent;
/** Multicast delegate which will broadcast a notification when the compiler finishes */
FModuleCompilerFinishedEvent ModuleCompilerFinishedEvent;
/** When compiling a module using an external application, stores the handle to the process that is running */
FProcHandle ModuleCompileProcessHandle;
/** When compiling a module using an external application, this is the process read pipe handle */
void* ModuleCompileReadPipe;
/** When compiling a module using an external application, this is the text that was read from the read pipe handle */
FString ModuleCompileReadPipeText;
/** Callback to execute after an asynchronous recompile has completed (whether successful or not.) */
FRecompileModulesCallback RecompileModulesCallback;
/** true if we should attempt to cancel the current async compilation */
bool bRequestCancelCompilation;
/** Tracks the validity of the game module existence */
EThreeStateBool::Type bIsAnyGameModuleLoaded;
/** True if the directory watcher has been successfully initialized */
bool bDirectoryWatcherInitialized;
/** Reconstructed CDOs map during hot-reload. */
TMap<UObject*, UObject*> ReconstructedCDOsMap;
/** Keeps record of hot-reload session starting time. */
double HotReloadStartTime;
};
namespace HotReloadDefs
{
const static FString CompilationInfoConfigSection("ModuleFileTracking");
// These strings should match the values of the enum EModuleCompileMethod in ModuleManager.h
// and should be handled in ReadModuleCompilationInfoFromConfig() & WriteModuleCompilationInfoToConfig() below
const static FString CompileMethodRuntime("Runtime");
const static FString CompileMethodExternal("External");
const static FString CompileMethodUnknown("Unknown");
// Add one minute epsilon to timestamp comparision
const static FTimespan TimeStampEpsilon(0, 1, 0);
}
IMPLEMENT_MODULE(FHotReloadModule, HotReload);
namespace UE4HotReload_Private
{
/**
* Gets editor runs directory.
*/
FString GetEditorRunsDir()
{
FString TempDir = FPaths::EngineIntermediateDir();
return FPaths::Combine(*TempDir, TEXT("EditorRuns"));
}
/**
* Creates a file that informs UBT that the editor is currently running.
*/
void CreateFileThatIndicatesEditorRunIfNeeded()
{
#if WITH_EDITOR
IPlatformFile& FS = IPlatformFile::GetPlatformPhysical();
FString EditorRunsDir = GetEditorRunsDir();
FString FileName = FPaths::Combine(*EditorRunsDir, *FString::Printf(TEXT("%d"), FPlatformProcess::GetCurrentProcessId()));
if (FS.FileExists(*FileName))
{
if (!GIsEditor)
{
FS.DeleteFile(*FileName);
}
}
else
{
if (GIsEditor)
{
if (!FS.CreateDirectory(*EditorRunsDir))
{
return;
}
delete FS.OpenWrite(*FileName); // Touch file.
}
}
#endif // WITH_EDITOR
}
/**
* Deletes file left by CreateFileThatIndicatesEditorRunIfNeeded function.
*/
void DeleteFileThatIndicatesEditorRunIfNeeded()
{
#if WITH_EDITOR
IPlatformFile& FS = IPlatformFile::GetPlatformPhysical();
FString EditorRunsDir = GetEditorRunsDir();
FString FileName = FPaths::Combine(*EditorRunsDir, *FString::Printf(TEXT("%d"), FPlatformProcess::GetCurrentProcessId()));
if (FS.FileExists(*FileName))
{
FS.DeleteFile(*FileName);
}
#endif // WITH_EDITOR
}
/**
* Gets all currently loaded game module names and optionally, the file names for those modules
*/
TArray<FString> GetGameModuleNames(const FModuleManager& ModuleManager)
{
TArray<FString> Result;
// Ask the module manager for a list of currently-loaded gameplay modules
TArray<FModuleStatus> ModuleStatuses;
ModuleManager.QueryModules(ModuleStatuses);
for (FModuleStatus& ModuleStatus : ModuleStatuses)
{
// We only care about game modules that are currently loaded
if (ModuleStatus.bIsLoaded && ModuleStatus.bIsGameModule)
{
Result.Add(MoveTemp(ModuleStatus.Name));
}
}
return Result;
}
/**
* Gets all currently loaded game module names and optionally, the file names for those modules
*/
TMap<FString, FString> GetGameModuleFilenames(const FModuleManager& ModuleManager)
{
TMap<FString, FString> Result;
// Ask the module manager for a list of currently-loaded gameplay modules
TArray< FModuleStatus > ModuleStatuses;
ModuleManager.QueryModules(ModuleStatuses);
for (FModuleStatus& ModuleStatus : ModuleStatuses)
{
// We only care about game modules that are currently loaded
if (ModuleStatus.bIsLoaded && ModuleStatus.bIsGameModule)
{
Result.Add(MoveTemp(ModuleStatus.Name), MoveTemp(ModuleStatus.FilePath));
}
}
return Result;
}
struct FPackagesAndDependentNames
{
TArray<UPackage*> Packages;
TArray<FName> DependentNames;
};
/**
* Gets named packages and the names dependents.
*/
FPackagesAndDependentNames SplitByPackagesAndDependentNames(const TArray<FString>& ModuleNames)
{
FPackagesAndDependentNames Result;
for (const FString& ModuleName : ModuleNames)
{
FString PackagePath = TEXT("/Script/") + ModuleName;
if (UPackage* Package = FindPackage(nullptr, *PackagePath))
{
Result.Packages.Add(Package);
}
else
{
Result.DependentNames.Add(*ModuleName);
}
}
return Result;
}
}
void FHotReloadModule::StartupModule()
{
UE4HotReload_Private::CreateFileThatIndicatesEditorRunIfNeeded();
bIsHotReloadingFromEditor = false;
#if WITH_ENGINE
// Register re-instancing delegate (Core)
FCoreUObjectDelegates::RegisterClassForHotReloadReinstancingDelegate.AddRaw(this, &FHotReloadModule::RegisterForReinstancing);
FCoreUObjectDelegates::ReinstanceHotReloadedClassesDelegate.AddRaw(this, &FHotReloadModule::ReinstanceClasses);
#endif
// Register directory watcher delegate
RefreshHotReloadWatcher();
// Register hot-reload from IDE ticker
TickerDelegate = FTickerDelegate::CreateRaw(this, &FHotReloadModule::Tick);
TickerDelegateHandle = FTicker::GetCoreTicker().AddTicker(TickerDelegate);
FModuleManager::Get().OnModulesChanged().AddRaw(this, &FHotReloadModule::ModulesChangedCallback);
IPluginManager::Get().OnNewPluginMounted().AddRaw(this, &FHotReloadModule::PluginMountedCallback);
}
void FHotReloadModule::ShutdownModule()
{
FTicker::GetCoreTicker().RemoveTicker(TickerDelegateHandle);
ShutdownHotReloadWatcher();
UE4HotReload_Private::DeleteFileThatIndicatesEditorRunIfNeeded();
}
bool FHotReloadModule::Exec( UWorld* Inworld, const TCHAR* Cmd, FOutputDevice& Ar )
{
#if !UE_BUILD_SHIPPING
if ( FParse::Command( &Cmd, TEXT( "Module" ) ) )
{
#if WITH_HOT_RELOAD
// Recompile <ModuleName>
if( FParse::Command( &Cmd, TEXT( "Recompile" ) ) )
{
const FString ModuleNameStr = FParse::Token( Cmd, 0 );
if( !ModuleNameStr.IsEmpty() )
{
const FName ModuleName( *ModuleNameStr );
const bool bReloadAfterRecompile = true;
const bool bForceCodeProject = false;
const bool bFailIfGeneratedCodeChanges = true;
RecompileModule( ModuleName, bReloadAfterRecompile, Ar, bFailIfGeneratedCodeChanges, bForceCodeProject);
}
return true;
}
#endif // WITH_HOT_RELOAD
}
#endif // !UE_BUILD_SHIPPING
return false;
}
void FHotReloadModule::Tick()
{
// We never want to block on a pending compile when checking compilation status during Tick(). We're
// just checking so that we can fire callbacks if and when compilation has finished.
const bool bWaitForCompletion = false;
// Ignored output variables
bool bCompileStillInProgress = false;
bool bCompileSucceeded = false;
FOutputDeviceNull NullOutput;
CheckForFinishedModuleDLLCompile( bWaitForCompletion, bCompileStillInProgress, bCompileSucceeded, NullOutput );
}
void FHotReloadModule::SaveConfig()
{
// Find all the modules
TArray<FModuleStatus> Modules;
FModuleManager::Get().QueryModules(Modules);
// Update the compile data for each one
for( const FModuleStatus &Module : Modules )
{
UpdateModuleCompileData(*Module.Name);
}
}
FString FHotReloadModule::GetModuleCompileMethod(FName InModuleName)
{
if (!ModuleCompileData.Contains(InModuleName))
{
UpdateModuleCompileData(InModuleName);
}
switch(ModuleCompileData.FindChecked(InModuleName).Get().CompileMethod)
{
case EModuleCompileMethod::External:
return HotReloadDefs::CompileMethodExternal;
case EModuleCompileMethod::Runtime:
return HotReloadDefs::CompileMethodRuntime;
default:
return HotReloadDefs::CompileMethodUnknown;
}
}
bool FHotReloadModule::RecompileModule(const FName InModuleName, const bool bReloadAfterRecompile, FOutputDevice &Ar, bool bFailIfGeneratedCodeChanges, bool bForceCodeProject)
{
#if WITH_HOT_RELOAD
UE_LOG(LogHotReload, Log, TEXT("Recompiling module %s..."), *InModuleName.ToString());
// This is an internal request for hot-reload (not from IDE)
bIsHotReloadingFromEditor = true;
// A list of modules that have been recompiled in the editor is going to prevent false
// hot-reload from IDE events as this call is blocking any potential callbacks coming from the filesystem
// and bIsHotReloadingFromEditor may not be enough to prevent those from being treated as actual hot-reload from IDE modules
ModulesRecentlyCompiledInTheEditor.Empty();
FFormatNamedArguments Args;
Args.Add( TEXT("CodeModuleName"), FText::FromName( InModuleName ) );
const FText StatusUpdate = FText::Format( NSLOCTEXT("ModuleManager", "Recompile_SlowTaskName", "Compiling {CodeModuleName}..."), Args );
FScopedSlowTask SlowTask(2, StatusUpdate);
SlowTask.MakeDialog();
ModuleCompilerStartedEvent.Broadcast(false); // we never perform an async compile
FModuleManager& ModuleManager = FModuleManager::Get();
// Update our set of known modules, in case we don't already know about this module
ModuleManager.AddModule( InModuleName );
// Only use rolling module names if the module was already loaded into memory. This allows us to try compiling
// the module without actually having to unload it first.
const bool bWasModuleLoaded = ModuleManager.IsModuleLoaded( InModuleName );
const bool bUseRollingModuleNames = bWasModuleLoaded;
SlowTask.EnterProgressFrame();
bool bWasSuccessful = true;
if( bUseRollingModuleNames )
{
// First, try to compile the module. If the module is already loaded, we won't unload it quite yet. Instead
// make sure that it compiles successfully.
// Find a unique file name for the module
FString UniqueSuffix;
FString UniqueModuleFileName;
ModuleManager.MakeUniqueModuleFilename( InModuleName, UniqueSuffix, UniqueModuleFileName );
TArray< FModuleToRecompile > ModulesToRecompile;
FModuleToRecompile ModuleToRecompile;
ModuleToRecompile.ModuleName = InModuleName.ToString();
ModuleToRecompile.ModuleFileSuffix = UniqueSuffix;
ModuleToRecompile.NewModuleFilename = UniqueModuleFileName;
ModulesToRecompile.Add( ModuleToRecompile );
ModulesRecentlyCompiledInTheEditor.Add(FPaths::ConvertRelativePathToFull(UniqueModuleFileName));
bWasSuccessful = RecompileModuleDLLs(ModulesToRecompile, Ar, bFailIfGeneratedCodeChanges, bForceCodeProject);
}
SlowTask.EnterProgressFrame();
if( bWasSuccessful )
{
// Shutdown the module if it's already running
if( bWasModuleLoaded )
{
Ar.Logf( TEXT( "Unloading module before compile." ) );
ModuleManager.UnloadOrAbandonModuleWithCallback( InModuleName, Ar );
}
if( !bUseRollingModuleNames )
{
// Try to recompile the DLL
TArray< FModuleToRecompile > ModulesToRecompile;
FModuleToRecompile ModuleToRecompile;
ModuleToRecompile.ModuleName = InModuleName.ToString();
if (ModuleManager.IsModuleLoaded(InModuleName))
{
ModulesRecentlyCompiledInTheEditor.Add(FPaths::ConvertRelativePathToFull(ModuleManager.GetModuleFilename(InModuleName)));
}
else
{
ModuleToRecompile.NewModuleFilename = ModuleManager.GetGameBinariesDirectory() / FModuleManager::GetCleanModuleFilename(InModuleName, true);
ModulesRecentlyCompiledInTheEditor.Add(FPaths::ConvertRelativePathToFull(ModuleToRecompile.NewModuleFilename));
}
ModulesToRecompile.Add( ModuleToRecompile );
bWasSuccessful = RecompileModuleDLLs(ModulesToRecompile, Ar, bFailIfGeneratedCodeChanges, bForceCodeProject);
}
// Reload the module if it was loaded before we recompiled
if( bWasSuccessful && (bWasModuleLoaded || bForceCodeProject) && bReloadAfterRecompile )
{
TGuardValue<bool> GuardIsHotReload(GIsHotReload, true);
Ar.Logf( TEXT( "Reloading module %s after successful compile." ), *InModuleName.ToString() );
bWasSuccessful = ModuleManager.LoadModuleWithCallback( InModuleName, Ar );
CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS);
}
}
if (bForceCodeProject && bWasSuccessful)
{
HotReloadEvent.Broadcast( false );
}
bIsHotReloadingFromEditor = false;
return bWasSuccessful;
#else
return false;
#endif // WITH_HOT_RELOAD
}
/** Type hash for a UObject Function Pointer, maybe not a great choice, but it should be sufficient for the needs here. **/
inline uint32 GetTypeHash(FNativeFuncPtr A)
{
return *(uint32*)&A;
}
/** Map from old function pointer to new function pointer for hot reload. */
static TMap<FNativeFuncPtr, FNativeFuncPtr> HotReloadFunctionRemap;
static TSet<UBlueprint*> HotReloadBPSetToRecompile;
static TSet<UBlueprint*> HotReloadBPSetToRecompileBytecodeOnly;
/** Adds and entry for the UFunction native pointer remap table */
void FHotReloadModule::AddHotReloadFunctionRemap(FNativeFuncPtr NewFunctionPointer, FNativeFuncPtr OldFunctionPointer)
{
FNativeFuncPtr OtherNewFunction = HotReloadFunctionRemap.FindRef(OldFunctionPointer);
check(!OtherNewFunction || OtherNewFunction == NewFunctionPointer);
check(NewFunctionPointer);
check(OldFunctionPointer);
HotReloadFunctionRemap.Add(OldFunctionPointer, NewFunctionPointer);
}
ECompilationResult::Type FHotReloadModule::DoHotReloadFromEditor(const bool bWaitForCompletion)
{
// Get all game modules we want to compile
const FModuleManager& ModuleManager = FModuleManager::Get();
TArray<FString> GameModuleNames = UE4HotReload_Private::GetGameModuleNames(ModuleManager);
int32 NumPackagesToRebind = 0;
int32 NumDependentModules = 0;
ECompilationResult::Type Result = ECompilationResult::Unsupported;
// Analytics
double Duration = 0.0;
if (GameModuleNames.Num() > 0)
{
FScopedDurationTimer Timer(Duration);
UE4HotReload_Private::FPackagesAndDependentNames PackagesAndDependentNames = UE4HotReload_Private::SplitByPackagesAndDependentNames(GameModuleNames);
NumPackagesToRebind = PackagesAndDependentNames.Packages.Num();
NumDependentModules = PackagesAndDependentNames.DependentNames.Num();
Result = RebindPackagesInternal(MoveTemp(PackagesAndDependentNames.Packages), MoveTemp(PackagesAndDependentNames.DependentNames), bWaitForCompletion, *GLog);
}
RecordAnalyticsEvent(TEXT("Editor"), Result, Duration, NumPackagesToRebind, NumDependentModules);
return Result;
}
ECompilationResult::Type FHotReloadModule::DoHotReloadInternal(const TMap<FString, FString>& ChangedModules, const TArray<UPackage*>& Packages, const TArray<FName>& InDependentModules, FOutputDevice& HotReloadAr)
{
#if WITH_HOT_RELOAD
FModuleManager& ModuleManager = FModuleManager::Get();
ModuleManager.ResetModulePathsCache();
FFeedbackContext& ErrorsFC = UClass::GetDefaultPropertiesFeedbackContext();
ErrorsFC.ClearWarningsAndErrors();
// Rebind the hot reload DLL
TGuardValue<bool> GuardIsHotReload(GIsHotReload, true);
TGuardValue<bool> GuardIsInitialLoad(GIsInitialLoad, true);
HotReloadFunctionRemap.Empty(); // redundant
CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS); // we create a new CDO in the transient package...this needs to go away before we try again.
// Load the new modules up
bool bReloadSucceeded = false;
ECompilationResult::Type Result = ECompilationResult::Unsupported;
for (UPackage* Package : Packages)
{
FString PackageName = Package->GetName();
FString ShortPackageName = FPackageName::GetShortName(PackageName);
if (!ChangedModules.Contains(ShortPackageName))
{
continue;
}
FName ShortPackageFName = *ShortPackageName;
// Abandon the old module. We can't unload it because various data structures may be living
// that have vtables pointing to code that would become invalidated.
ModuleManager.AbandonModuleWithCallback(ShortPackageFName);
// Load the newly-recompiled module up (it will actually have a different DLL file name at this point.)
bReloadSucceeded = ModuleManager.LoadModule(ShortPackageFName) != nullptr;
if (!bReloadSucceeded)
{
HotReloadAr.Logf(ELogVerbosity::Warning, TEXT("HotReload failed, reload failed %s."), *PackageName);
Result = ECompilationResult::OtherCompilationError;
break;
}
}
// Load dependent modules.
for (FName ModuleName : InDependentModules)
{
FString ModuleNameStr = ModuleName.ToString();
if (!ChangedModules.Contains(ModuleNameStr))
{
continue;
}
ModuleManager.UnloadOrAbandonModuleWithCallback(ModuleName, HotReloadAr);
const bool bLoaded = ModuleManager.LoadModuleWithCallback(ModuleName, HotReloadAr);
if (!bLoaded)
{
HotReloadAr.Logf(ELogVerbosity::Warning, TEXT("Unable to reload module %s"), *ModuleName.GetPlainNameString());
}
}
if (ErrorsFC.GetNumErrors() || ErrorsFC.GetNumWarnings())
{
TArray<FString> AllErrorsAndWarnings;
ErrorsFC.GetErrorsAndWarningsAndEmpty(AllErrorsAndWarnings);
FString AllInOne;
for (const FString& ErrorOrWarning : AllErrorsAndWarnings)
{
AllInOne += ErrorOrWarning;
AllInOne += TEXT("\n");
}
HotReloadAr.Logf(ELogVerbosity::Warning, TEXT("Some classes could not be reloaded:\n%s"), *AllInOne);
}
if (bReloadSucceeded)
{
int32 NumFunctionsRemapped = 0;
// Remap all native functions (and gather scriptstructs)
TArray<UScriptStruct*> ScriptStructs;
for (FRawObjectIterator It; It; ++It)
{
if (UFunction* Function = Cast<UFunction>(static_cast<UObject*>(It->Object)))
{
if (FNativeFuncPtr NewFunction = HotReloadFunctionRemap.FindRef(Function->GetNativeFunc()))
{
++NumFunctionsRemapped;
Function->SetNativeFunc(NewFunction);
}
}
if (UScriptStruct* ScriptStruct = Cast<UScriptStruct>(static_cast<UObject*>(It->Object)))
{
if (Packages.ContainsByPredicate([=](UPackage* Package) { return ScriptStruct->IsIn(Package); }) && !ScriptStruct->HasAnyFlags(RF_ClassDefaultObject) && ScriptStruct->GetCppStructOps())
{
ScriptStructs.Add(ScriptStruct);
}
}
}
// now let's set up the script structs...this relies on super behavior, so null them all, then set them all up. Internally this sets them up hierarchically.
for (UScriptStruct* Script : ScriptStructs)
{
Script->ClearCppStructOps();
}
for (UScriptStruct* Script : ScriptStructs)
{
Script->PrepareCppStructOps();
check(Script->GetCppStructOps());
}
// Make sure new classes have the token stream assembled
UClass::AssembleReferenceTokenStreams();
HotReloadAr.Logf(ELogVerbosity::Display, TEXT("HotReload successful (%d functions remapped %d scriptstructs remapped)"), NumFunctionsRemapped, ScriptStructs.Num());
HotReloadFunctionRemap.Empty();
ReplaceReferencesToReconstructedCDOs();
// Force GC to collect reinstanced objects
CollectGarbage(GARBAGE_COLLECTION_KEEPFLAGS, true);
Result = ECompilationResult::Succeeded;
}
HotReloadEvent.Broadcast( !bIsHotReloadingFromEditor );
HotReloadAr.Logf(ELogVerbosity::Display, TEXT("HotReload took %4.1fs."), FPlatformTime::Seconds() - HotReloadStartTime);
bIsHotReloadingFromEditor = false;
return Result;
#else
bIsHotReloadingFromEditor = false;
return ECompilationResult::Unsupported;
#endif
}
void FHotReloadModule::ReplaceReferencesToReconstructedCDOs()
{
if (ReconstructedCDOsMap.Num() == 0)
{
return;
}
// Thread pool manager. We need new thread pool with increased
// amount of stack size. Standard GThreadPool was encountering
// stack overflow error during serialization.
static struct FReplaceReferencesThreadPool
{
FReplaceReferencesThreadPool()
{
Pool = FQueuedThreadPool::Allocate();
int32 NumThreadsInThreadPool = FPlatformMisc::NumberOfWorkerThreadsToSpawn();
verify(Pool->Create(NumThreadsInThreadPool, 256 * 1024));
}
~FReplaceReferencesThreadPool()
{
Pool->Destroy();
}
FQueuedThreadPool* GetPool() { return Pool; }
private:
FQueuedThreadPool* Pool;
} ThreadPoolManager;
// Async task to enable multithreaded CDOs reference search.
class FFindRefTask : public FNonAbandonableTask
{
public:
explicit FFindRefTask(const TMap<UObject*, UObject*>& InReconstructedCDOsMap, int32 ReserveElements)
: ReconstructedCDOsMap(InReconstructedCDOsMap)
{
ObjectsArray.Reserve(ReserveElements);
}
void DoWork()
{
for (UObject* Object : ObjectsArray)
{
class FReplaceCDOReferencesArchive : public FArchiveUObject
{
public:
FReplaceCDOReferencesArchive(UObject* InPotentialReferencer, const TMap<UObject*, UObject*>& InReconstructedCDOsMap)
: ReconstructedCDOsMap(InReconstructedCDOsMap)
, PotentialReferencer(InPotentialReferencer)
{
ArIsObjectReferenceCollector = true;
ArIgnoreOuterRef = true;
}
virtual FString GetArchiveName() const override
{
return TEXT("FReplaceCDOReferencesArchive");
}
FArchive& operator<<(UObject*& ObjRef)
{
UObject* Obj = ObjRef;
if (Obj && Obj != PotentialReferencer)
{
if (UObject* const* FoundObj = ReconstructedCDOsMap.Find(Obj))
{
ObjRef = *FoundObj;
}
}
return *this;
}
const TMap<UObject*, UObject*>& ReconstructedCDOsMap;
UObject* PotentialReferencer;
};
FReplaceCDOReferencesArchive FindRefsArchive(Object, ReconstructedCDOsMap);
Object->Serialize(FindRefsArchive);
}
}
FORCEINLINE TStatId GetStatId() const
{
RETURN_QUICK_DECLARE_CYCLE_STAT(FFindRefTask, STATGROUP_ThreadPoolAsyncTasks);
}
TArray<UObject*> ObjectsArray;
private:
const TMap<UObject*, UObject*>& ReconstructedCDOsMap;
};
const int32 NumberOfThreads = FPlatformMisc::NumberOfWorkerThreadsToSpawn();
const int32 NumObjects = GUObjectArray.GetObjectArrayNum();
const int32 ObjectsPerTask = FMath::CeilToInt((float)NumObjects / NumberOfThreads);
// Create tasks.
TArray<FAsyncTask<FFindRefTask>> Tasks;
Tasks.Reserve(NumberOfThreads);
for (int32 TaskId = 0; TaskId < NumberOfThreads; ++TaskId)
{
Tasks.Emplace(ReconstructedCDOsMap, ObjectsPerTask);
}
// Distribute objects uniformly between tasks.
int32 CurrentTaskId = 0;
for (FObjectIterator ObjIter; ObjIter; ++ObjIter)
{
UObject* CurObject = *ObjIter;
if (CurObject->IsPendingKill())
{
continue;
}
Tasks[CurrentTaskId].GetTask().ObjectsArray.Add(CurObject);
CurrentTaskId = (CurrentTaskId + 1) % NumberOfThreads;
}
// Run async tasks in worker threads.
for (FAsyncTask<FFindRefTask>& Task : Tasks)
{
Task.StartBackgroundTask(ThreadPoolManager.GetPool());
}
// Wait until tasks are finished
for (FAsyncTask<FFindRefTask>& AsyncTask : Tasks)
{
AsyncTask.EnsureCompletion();
}
ReconstructedCDOsMap.Empty();
}
ECompilationResult::Type FHotReloadModule::RebindPackages(TArray<UPackage*> InPackages, TArray<FName> DependentModules, const bool bWaitForCompletion, FOutputDevice &Ar)
{
ECompilationResult::Type Result = ECompilationResult::Unknown;
// Get game packages
const FModuleManager& ModuleManager = FModuleManager::Get();
TArray<FString> GameModuleNames = UE4HotReload_Private::GetGameModuleNames(ModuleManager);
UE4HotReload_Private::FPackagesAndDependentNames PackagesAndDependentNames = UE4HotReload_Private::SplitByPackagesAndDependentNames(GameModuleNames);
// Get a set of source packages combined with game packages
TSet<UPackage*> PackagesIncludingGame(InPackages);
int32 NumInPackages = PackagesIncludingGame.Num();
PackagesIncludingGame.Append(PackagesAndDependentNames.Packages);
// Check if there was any overlap
bool bInPackagesIncludeGame = PackagesIncludingGame.Num() < NumInPackages + PackagesAndDependentNames.Packages.Num();
// If any of those modules were game modules, we'll compile those too
TArray<UPackage*> Packages;
TArray<FName> Dependencies;
if (bInPackagesIncludeGame)
{
Packages = PackagesIncludingGame.Array();
Dependencies = MoveTemp(PackagesAndDependentNames.DependentNames);
}
else
{
Packages = InPackages;
}
double Duration = 0.0;
int32 NumPackages = InPackages.Num();
int32 NumDependentModules = DependentModules.Num();
{
FScopedDurationTimer RebindTimer(Duration);
Result = RebindPackagesInternal(MoveTemp(Packages), MoveTemp(Dependencies), bWaitForCompletion, Ar);
}
RecordAnalyticsEvent(TEXT("Rebind"), Result, Duration, Packages.Num(), Dependencies.Num());
return Result;
}
ECompilationResult::Type FHotReloadModule::RebindPackagesInternal(TArray<UPackage*>&& InPackages, TArray<FName>&& DependentModules, const bool bWaitForCompletion, FOutputDevice& Ar)
{
ECompilationResult::Type Result = ECompilationResult::Unsupported;
#if WITH_HOT_RELOAD
bool bCanRebind = InPackages.Num() > 0;
// Verify that we're going to be able to rebind the specified packages
if (bCanRebind)
{
for (UPackage* Package : InPackages)
{
check(Package);
if (Package->GetOuter() != NULL)
{
Ar.Logf(ELogVerbosity::Warning, TEXT("Could not rebind package for %s, package is either not bound yet or is not a DLL."), *Package->GetName());
bCanRebind = false;
break;
}
}
}
// We can only proceed if a compile isn't already in progress
if (IsCurrentlyCompiling())
{
Ar.Logf(ELogVerbosity::Warning, TEXT("Could not rebind package because a module compile is already in progress."));
bCanRebind = false;
}
if (bCanRebind)
{
FModuleManager::Get().ResetModulePathsCache();
bIsHotReloadingFromEditor = true;
HotReloadStartTime = FPlatformTime::Seconds();
TArray< FName > ModuleNames;
for (UPackage* Package : InPackages)
{
// Attempt to recompile this package's module
FName ShortPackageName = FPackageName::GetShortFName(Package->GetFName());
ModuleNames.Add(ShortPackageName);
}
// Add dependent modules.
ModuleNames.Append(DependentModules);
// Start compiling modules
const bool bCompileStarted = RecompileModulesAsync(
ModuleNames,
[this, InPackages/*=MoveTemp(InPackages)*/, DependentModules/*=MoveTemp(DependentModules)*/, &Ar](const TMap<FString, FString>& ChangedModules, bool bRecompileFinished, ECompilationResult::Type CompilationResult)
{
if (ECompilationResult::Failed(CompilationResult) && bRecompileFinished)
{
Ar.Logf(ELogVerbosity::Warning, TEXT("HotReload failed, recompile failed"));
return;
}
DoHotReloadInternal(ChangedModules, InPackages, DependentModules, Ar);
},
bWaitForCompletion,
Ar);
if (bCompileStarted)
{
if (bWaitForCompletion)
{
Ar.Logf(ELogVerbosity::Warning, TEXT("HotReload operation took %4.1fs."), float(FPlatformTime::Seconds() - HotReloadStartTime));
bIsHotReloadingFromEditor = false;
}
else
{
Ar.Logf(ELogVerbosity::Warning, TEXT("Starting HotReload took %4.1fs."), float(FPlatformTime::Seconds() - HotReloadStartTime));
}
Result = ECompilationResult::Succeeded;
}
else
{
Ar.Logf(ELogVerbosity::Warning, TEXT("RebindPackages failed because the compiler could not be started."));
Result = ECompilationResult::OtherCompilationError;
bIsHotReloadingFromEditor = false;
}
}
else
#endif
{
Ar.Logf(ELogVerbosity::Warning, TEXT("RebindPackages not possible for specified packages (or application was compiled in monolithic mode.)"));
}
return Result;
}
#if WITH_ENGINE
namespace {
static TArray<TPair<UClass*, UClass*> >& GetClassesToReinstance()
{
static TArray<TPair<UClass*, UClass*> > Data;
return Data;
}
}
void FHotReloadModule::RegisterForReinstancing(UClass* OldClass, UClass* NewClass)
{
TPair<UClass*, UClass*> Pair;
Pair.Key = OldClass;
Pair.Value = NewClass;
TArray<TPair<UClass*, UClass*> >& ClassesToReinstance = GetClassesToReinstance();
ClassesToReinstance.Add(MoveTemp(Pair));
}
void FHotReloadModule::ReinstanceClasses()
{
#if WITH_HOT_RELOAD
if (GIsHotReload)
{
UClass::AssembleReferenceTokenStreams();
}
#endif // WITH_HOT_RELOAD
TArray<TPair<UClass*, UClass*> >& ClassesToReinstance = GetClassesToReinstance();
TMap<UClass*, UClass*> OldToNewClassesMap;
for (const TPair<UClass*, UClass*>& Pair : ClassesToReinstance)
{
// Don't allow reinstancing of UEngine classes
if (Pair.Key->IsChildOf(UEngine::StaticClass()))
{
UE_LOG(LogHotReload, Warning, TEXT("Engine class '%s' has changed but will be ignored for hot reload"), *Pair.Key->GetName());
continue;
}
if (Pair.Value != nullptr)
{
OldToNewClassesMap.Add(Pair.Key, Pair.Value);
}
}
for (const TPair<UClass*, UClass*>& Pair : ClassesToReinstance)
{
// Don't allow reinstancing of UEngine classes
if (!Pair.Key->IsChildOf(UEngine::StaticClass()))
{
ReinstanceClass(Pair.Key, Pair.Value, OldToNewClassesMap);
}
}
ClassesToReinstance.Empty();
}
void FHotReloadModule::ReinstanceClass(UClass* OldClass, UClass* NewClass, const TMap<UClass*, UClass*>& OldToNewClassesMap)
{
TSharedPtr<FHotReloadClassReinstancer> ReinstanceHelper = FHotReloadClassReinstancer::Create(NewClass, OldClass, OldToNewClassesMap, ReconstructedCDOsMap, HotReloadBPSetToRecompile, HotReloadBPSetToRecompileBytecodeOnly);
if (ReinstanceHelper->ClassNeedsReinstancing())
{
UE_LOG(LogHotReload, Log, TEXT("Re-instancing %s after hot-reload."), NewClass ? *NewClass->GetName() : *OldClass->GetName());
ReinstanceHelper->ReinstanceObjectsAndUpdateDefaults();
}
}
#endif
void FHotReloadModule::OnHotReloadBinariesChanged(const TArray<struct FFileChangeData>& FileChanges)
{
if (bIsHotReloadingFromEditor)
{
// DO NOTHING, this case is handled by RebindPackages
return;
}
const FModuleManager& ModuleManager = FModuleManager::Get();
TMap<FString, FString> GameModuleFilenames = UE4HotReload_Private::GetGameModuleFilenames(ModuleManager);
if (GameModuleFilenames.Num() == 0)
{
return;
}
// Check if any of the game DLLs has been added
for (auto& Change : FileChanges)
{
// Ignore changes that aren't introducing a new file.
//
// On the Mac the Add event is for a temporary linker(?) file that gets immediately renamed
// to a dylib. In the future we may want to support modified event for all platforms anyway once
// shadow copying works with hot-reload.
#if PLATFORM_MAC
if (Change.Action != FFileChangeData::FCA_Modified)
#else
if (Change.Action != FFileChangeData::FCA_Added)
#endif
{
continue;
}
// Ignore files that aren't of module type
FString Filename = FPaths::GetCleanFilename(Change.Filename);
if (!Filename.EndsWith(FPlatformProcess::GetModuleExtension()))
{
continue;
}
for (const TPair<FString, FString>& NameFilename : GameModuleFilenames)
{
// Handle module files which have already been hot-reloaded.
FString BaseName = FPaths::GetBaseFilename(NameFilename.Value);
StripModuleSuffixFromFilename(BaseName, NameFilename.Key);
// Hot reload always adds a numbered suffix preceded by a hyphen, but otherwise the module name must match exactly!
if (!Filename.StartsWith(BaseName + TEXT("-")))
{
continue;
}
if (NewModules.Contains(NameFilename.Key))
{
continue;
}
if (ModulesRecentlyCompiledInTheEditor.Contains(FPaths::ConvertRelativePathToFull(Change.Filename)))
{
continue;
}
// Add to queue. We do not hot-reload here as there may potentially be other modules being compiled.
NewModules.Emplace(NameFilename.Key, Change.Filename);
UE_LOG(LogHotReload, Log, TEXT("New module detected: %s"), *Filename);
}
}
}
void FHotReloadModule::StripModuleSuffixFromFilename(FString& InOutModuleFilename, const FString& ModuleName)
{
// First hyphen is where the UE4Edtior prefix ends
int32 FirstHyphenIndex = INDEX_NONE;
if (InOutModuleFilename.FindChar('-', FirstHyphenIndex))
{
// Second hyphen means we already have a hot-reloaded module or other than Development config module
int32 SecondHyphenIndex = FirstHyphenIndex;
do
{
SecondHyphenIndex = InOutModuleFilename.Find(TEXT("-"), ESearchCase::IgnoreCase, ESearchDir::FromStart, SecondHyphenIndex + 1);
if (SecondHyphenIndex != INDEX_NONE)
{
// Make sure that the section between hyphens is the expected module name. This guards against cases where module name has a hyphen inside.
FString HotReloadedModuleName = InOutModuleFilename.Mid(FirstHyphenIndex + 1, SecondHyphenIndex - FirstHyphenIndex - 1);
if (HotReloadedModuleName == ModuleName)
{
InOutModuleFilename = InOutModuleFilename.Mid(0, SecondHyphenIndex);
SecondHyphenIndex = INDEX_NONE;
}
}
} while (SecondHyphenIndex != INDEX_NONE);
}
}
void FHotReloadModule::RefreshHotReloadWatcher()
{
FDirectoryWatcherModule& DirectoryWatcherModule = FModuleManager::Get().LoadModuleChecked<FDirectoryWatcherModule>(TEXT("DirectoryWatcher"));
IDirectoryWatcher* DirectoryWatcher = DirectoryWatcherModule.Get();
if (DirectoryWatcher)
{
// Watch the game directory
AddHotReloadDirectory(DirectoryWatcher, FPaths::ProjectDir());
// Also watch all the game plugin directories
for(const TSharedRef<IPlugin>& Plugin : IPluginManager::Get().GetEnabledPlugins())
{
if (Plugin->GetLoadedFrom() == EPluginLoadedFrom::Project && Plugin->GetDescriptor().Modules.Num() > 0)
{
AddHotReloadDirectory(DirectoryWatcher, Plugin->GetBaseDir());
}
}
}
}
void FHotReloadModule::AddHotReloadDirectory(IDirectoryWatcher* DirectoryWatcher, const FString& BaseDir)
{
FString BinariesPath = FPaths::ConvertRelativePathToFull(BaseDir / TEXT("Binaries") / FPlatformProcess::GetBinariesSubdirectory());
if (FPaths::DirectoryExists(BinariesPath) && !BinariesFolderChangedDelegateHandles.Contains(BinariesPath))
{
IDirectoryWatcher::FDirectoryChanged BinariesFolderChangedDelegate = IDirectoryWatcher::FDirectoryChanged::CreateRaw(this, &FHotReloadModule::OnHotReloadBinariesChanged);
FDelegateHandle Handle;
if (DirectoryWatcher->RegisterDirectoryChangedCallback_Handle(BinariesPath, BinariesFolderChangedDelegate, Handle))
{
BinariesFolderChangedDelegateHandles.Add(BinariesPath, Handle);
}
}
}
void FHotReloadModule::ShutdownHotReloadWatcher()
{
FDirectoryWatcherModule* DirectoryWatcherModule = FModuleManager::GetModulePtr<FDirectoryWatcherModule>(TEXT("DirectoryWatcher"));
if( DirectoryWatcherModule != nullptr )
{
IDirectoryWatcher* DirectoryWatcher = DirectoryWatcherModule->Get();
if (DirectoryWatcher)
{
for (const TPair<FString, FDelegateHandle>& Pair : BinariesFolderChangedDelegateHandles)
{
DirectoryWatcher->UnregisterDirectoryChangedCallback_Handle(Pair.Key, Pair.Value);
}
}
}
}
bool FHotReloadModule::Tick(float DeltaTime)
{
if (NewModules.Num())
{
#if WITH_EDITOR
if (GEditor)
{
// Don't allow hot reloading if we're running networked PIE instances
// The reason, is it's fairly complicated to handle the re-wiring that needs to happen when we re-instance objects like player controllers, possessed pawns, etc...
const TIndirectArray<FWorldContext>& WorldContextList = GEditor->GetWorldContexts();
for (const FWorldContext& WorldContext : WorldContextList)
{
if (WorldContext.World() && WorldContext.World()->WorldType == EWorldType::PIE && WorldContext.World()->NetDriver)
{
return true; // Don't allow automatic hot reloading if we're running PIE instances
}
}
}
#endif // WITH_EDITOR
// We have new modules in the queue, but make sure UBT has finished compiling all of them
if (!FDesktopPlatformModule::Get()->IsUnrealBuildToolRunning())
{
DoHotReloadFromIDE();
NewModules.Empty();
}
else
{
UE_LOG(LogHotReload, Verbose, TEXT("Detected %d reloaded modules but UnrealBuildTool is still running"), NewModules.Num());
}
}
return true;
}
void FHotReloadModule::DoHotReloadFromIDE()
{
const FModuleManager& ModuleManager = FModuleManager::Get();
IFileManager& FileManager = IFileManager::Get();
int32 NumPackagesToRebind = 0;
int32 NumDependentModules = 0;
ECompilationResult::Type Result = ECompilationResult::Unsupported;
double Duration = 0.0;
TArray<FString> GameModuleNames = UE4HotReload_Private::GetGameModuleNames(ModuleManager);
if (GameModuleNames.Num() > 0)
{
FScopedDurationTimer Timer(Duration);
// Remove any modules whose files have disappeared - this can happen if a compile event has
// failed and deleted a DLL that was there previously.
for (auto It = NewModules.CreateIterator(); It; ++It)
{
if (!FileManager.FileExists(*It->Value))
{
It.RemoveCurrent();
}
}
if (NewModules.Num() == 0)
{
return;
}
UE_LOG(LogHotReload, Log, TEXT("Starting Hot-Reload from IDE"));
HotReloadStartTime = FPlatformTime::Seconds();
FScopedSlowTask SlowTask(100.f, LOCTEXT("CompilingGameCode", "Compiling Game Code"));
SlowTask.MakeDialog();
// Update compile data before we start compiling
for (const TPair<FString, FString>& NewModule : NewModules)
{
// Move on 10% / num items
SlowTask.EnterProgressFrame(10.f/NewModules.Num());
FName ModuleName = *NewModule.Key;
UpdateModuleCompileData(ModuleName);
OnModuleCompileSucceeded(ModuleName, NewModule.Value);
}
SlowTask.EnterProgressFrame(10);
UE4HotReload_Private::FPackagesAndDependentNames PackagesAndDependentNames = UE4HotReload_Private::SplitByPackagesAndDependentNames(GameModuleNames);
SlowTask.EnterProgressFrame(80);
NumPackagesToRebind = PackagesAndDependentNames.Packages.Num();
NumDependentModules = PackagesAndDependentNames.DependentNames.Num();
Result = DoHotReloadInternal(NewModules, PackagesAndDependentNames.Packages, PackagesAndDependentNames.DependentNames, *GLog);
}
RecordAnalyticsEvent(TEXT("IDE"), Result, Duration, NumPackagesToRebind, NumDependentModules);
}
void FHotReloadModule::RecordAnalyticsEvent(const TCHAR* ReloadFrom, ECompilationResult::Type Result, double Duration, int32 PackageCount, int32 DependentModulesCount)
{
#if WITH_ENGINE
if (FEngineAnalytics::IsAvailable())
{
TArray< FAnalyticsEventAttribute > ReloadAttribs;
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("ReloadFrom"), ReloadFrom));
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("Result"), ECompilationResult::ToString(Result)));
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("Duration"), FString::Printf(TEXT("%.4lf"), Duration)));
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("Packages"), FString::Printf(TEXT("%d"), PackageCount)));
ReloadAttribs.Add(FAnalyticsEventAttribute(TEXT("DependentModules"), FString::Printf(TEXT("%d"), DependentModulesCount)));
FEngineAnalytics::GetProvider().RecordEvent(TEXT("Editor.Usage.HotReload"), ReloadAttribs);
}
#endif
}
bool FHotReloadModule::RecompileModulesAsync( const TArray< FName > ModuleNames, FRecompileModulesCallback&& InRecompileModulesCallback, const bool bWaitForCompletion, FOutputDevice &Ar )
{
#if WITH_HOT_RELOAD
// NOTE: This method of recompiling always using a rolling file name scheme, since we never want to unload before
// we start recompiling, and we need the output DLL to be unlocked before we invoke the compiler
ModuleCompilerStartedEvent.Broadcast(!bWaitForCompletion); // we perform an async compile providing we're not waiting for completion
FModuleManager& ModuleManager = FModuleManager::Get();
TArray< FModuleToRecompile > ModulesToRecompile;
for( FName CurModuleName : ModuleNames )
{
// Update our set of known modules, in case we don't already know about this module
ModuleManager.AddModule( CurModuleName );
// Find a unique file name for the module
FModuleToRecompile ModuleToRecompile;
ModuleToRecompile.ModuleName = CurModuleName.ToString();
ModuleManager.MakeUniqueModuleFilename( CurModuleName, ModuleToRecompile.ModuleFileSuffix, ModuleToRecompile.NewModuleFilename );
ModulesToRecompile.Add( ModuleToRecompile );
}
// Kick off compilation!
const FString AdditionalArguments = MakeUBTArgumentsForModuleCompiling();
const bool bFailIfGeneratedCodeChanges = false;
const bool bForceCodeProject = false;
bool bWasSuccessful = StartCompilingModuleDLLs(FApp::GetProjectName(), ModulesToRecompile, MoveTemp(InRecompileModulesCallback), Ar, bFailIfGeneratedCodeChanges, AdditionalArguments, bForceCodeProject);
if (bWasSuccessful)
{
// Go ahead and check for completion right away. This is really just so that we can handle the case
// where the user asked us to wait for the compile to finish before returning.
bool bCompileStillInProgress = false;
bool bCompileSucceeded = false;
FOutputDeviceNull NullOutput;
CheckForFinishedModuleDLLCompile( bWaitForCompletion, bCompileStillInProgress, bCompileSucceeded, NullOutput );
if( !bCompileStillInProgress && !bCompileSucceeded )
{
bWasSuccessful = false;
}
}
return bWasSuccessful;
#else
return false;
#endif // WITH_HOT_RELOAD
}
void FHotReloadModule::OnModuleCompileSucceeded(FName ModuleName, const FString& NewModuleFilename)
{
// If the compile succeeded, update the module info entry with the new file name for this module
FModuleManager::Get().SetModuleFilename(ModuleName, NewModuleFilename);
#if WITH_HOT_RELOAD
// UpdateModuleCompileData() should have been run before compiling so the
// data in ModuleInfo should be correct for the pre-compile dll file.
FModuleCompilationData& CompileData = ModuleCompileData.FindChecked(ModuleName).Get();
FDateTime FileTimeStamp;
bool bGotFileTimeStamp = GetModuleFileTimeStamp(ModuleName, FileTimeStamp);
CompileData.bHasFileTimeStamp = bGotFileTimeStamp;
CompileData.FileTimeStamp = FileTimeStamp;
if (CompileData.bHasFileTimeStamp)
{
CompileData.CompileMethod = EModuleCompileMethod::Runtime;
}
else
{
CompileData.CompileMethod = EModuleCompileMethod::Unknown;
}
WriteModuleCompilationInfoToConfig(ModuleName, CompileData);
#endif
}
bool FHotReloadModule::RecompileModuleDLLs(const TArray< FModuleToRecompile >& ModuleNames, FOutputDevice& Ar, bool bFailIfGeneratedCodeChanges, bool bForceCodeProject)
{
bool bCompileSucceeded = false;
#if WITH_HOT_RELOAD
const FString AdditionalArguments = MakeUBTArgumentsForModuleCompiling();
if (StartCompilingModuleDLLs(FApp::GetProjectName(), ModuleNames, nullptr, Ar, bFailIfGeneratedCodeChanges, AdditionalArguments, bForceCodeProject))
{
const bool bWaitForCompletion = true; // Always wait
bool bCompileStillInProgress = false;
CheckForFinishedModuleDLLCompile( bWaitForCompletion, bCompileStillInProgress, bCompileSucceeded, Ar );
}
#endif
return bCompileSucceeded;
}
FString FHotReloadModule::MakeUBTArgumentsForModuleCompiling()
{
FString AdditionalArguments;
if ( FPaths::IsProjectFilePathSet() )
{
// We have to pass FULL paths to UBT
FString FullProjectPath = FPaths::ConvertRelativePathToFull(FPaths::GetProjectFilePath());
// @todo projectdirs: Currently non-installed projects that exist under the UE4 root are compiled by UBT with no .uproject file
// name passed in (see bIsProjectTarget in VCProject.cs), which causes intermediate libraries to be saved to the Engine
// intermediate folder instead of the project's intermediate folder. We're emulating this behavior here for module
// recompiling, so that compiled modules will be able to find their import libraries in the original folder they were compiled.
if( FApp::IsEngineInstalled() || !FullProjectPath.StartsWith( FPaths::ConvertRelativePathToFull( FPaths::RootDir() ) ) )
{
const FString ProjectFilenameWithQuotes = FString::Printf(TEXT("\"%s\""), *FullProjectPath);
AdditionalArguments += FString::Printf(TEXT("%s "), *ProjectFilenameWithQuotes);
}
}
// Use new FastPDB option to cut down linking time. Currently disabled due to problems with missing symbols in VS2015.
// AdditionalArguments += TEXT(" -FastPDB");
return AdditionalArguments;
}
bool FHotReloadModule::StartCompilingModuleDLLs(const FString& GameName, const TArray< FModuleToRecompile >& ModuleNames,
FRecompileModulesCallback&& InRecompileModulesCallback, FOutputDevice& Ar, bool bInFailIfGeneratedCodeChanges,
const FString& InAdditionalCmdLineArgs, bool bForceCodeProject)
{
#if WITH_HOT_RELOAD
// Keep track of what we're compiling
ModulesBeingCompiled = ModuleNames;
ModulesThatWereBeingRecompiled = ModulesBeingCompiled;
const TCHAR* BuildPlatformName = FPlatformMisc::GetUBTPlatform();
const TCHAR* BuildConfigurationName = FModuleManager::GetUBTConfiguration();
RecompileModulesCallback = MoveTemp(InRecompileModulesCallback);
// Pass a module file suffix to UBT if we have one
FString ModuleArg;
if (ModuleNames.Num())
{
Ar.Logf(TEXT("Candidate modules for hot reload:"));
for( const FModuleToRecompile& Module : ModuleNames )
{
if( !Module.ModuleFileSuffix.IsEmpty() )
{
ModuleArg += FString::Printf( TEXT( " -ModuleWithSuffix %s %s" ), *Module.ModuleName, *Module.ModuleFileSuffix );
}
else
{
ModuleArg += FString::Printf( TEXT( " -Module %s" ), *Module.ModuleName );
}
Ar.Logf( TEXT( " %s" ), *Module.ModuleName );
// prepare the compile info in the FModuleInfo so that it can be compared after compiling
FName ModuleFName(*Module.ModuleName);
UpdateModuleCompileData(ModuleFName);
}
}
FString ExtraArg;
#if UE_EDITOR
// NOTE: When recompiling from the editor, we're passed the game target name, not the editor target name, but we'll
// pass "-editorrecompile" to UBT which tells UBT to figure out the editor target to use for this game, since
// we can't possibly know what the target is called from within the engine code.
ExtraArg = TEXT( "-editorrecompile " );
#endif
if (bInFailIfGeneratedCodeChanges)
{
// Additional argument to let UHT know that we can only compile the module if the generated code didn't change
ExtraArg += TEXT( "-FailIfGeneratedCodeChanges " );
}
// If there's nothing to compile, don't bother linking the DLLs as the old ones are up-to-date
ExtraArg += TEXT("-canskiplink ");
// Shared PCH does no work with hot-reloading engine/editor modules as we don't scan all modules for them.
if (!ContainsOnlyGameModules(ModuleNames))
{
ExtraArg += TEXT("-nosharedpch ");
}
FString TargetName = GameName;
#if WITH_EDITOR
// If there are no game modules loaded, then it's not a code-based project and the target
// for UBT should be the editor.
if (!bForceCodeProject && !IsAnyGameModuleLoaded())
{
TargetName = TEXT("UE4Editor");
}
#endif
FString CmdLineParams = FString::Printf( TEXT( "%s%s %s %s %s%s" ),
*TargetName, *ModuleArg,
BuildPlatformName, BuildConfigurationName,
*ExtraArg, *InAdditionalCmdLineArgs );
const bool bInvocationSuccessful = InvokeUnrealBuildToolForCompile(CmdLineParams, Ar);
if ( !bInvocationSuccessful )
{
// No longer compiling modules
ModulesBeingCompiled.Empty();
ModuleCompilerFinishedEvent.Broadcast(FString(), ECompilationResult::OtherCompilationError, false);
// Fire task completion delegate
if (RecompileModulesCallback)
{
RecompileModulesCallback( TMap<FString, FString>(), false, ECompilationResult::OtherCompilationError );
RecompileModulesCallback = nullptr;
}
}
return bInvocationSuccessful;
#else
return false;
#endif
}
bool FHotReloadModule::InvokeUnrealBuildToolForCompile(const FString& InCmdLineParams, FOutputDevice &Ar)
{
#if WITH_HOT_RELOAD
// Make sure we're not already compiling something!
check(!IsCurrentlyCompiling());
// Setup output redirection pipes, so that we can harvest compiler output and display it ourselves
void* PipeRead = NULL;
void* PipeWrite = NULL;
verify(FPlatformProcess::CreatePipe(PipeRead, PipeWrite));
ModuleCompileReadPipeText = TEXT("");
FProcHandle ProcHandle = FDesktopPlatformModule::Get()->InvokeUnrealBuildToolAsync(InCmdLineParams, Ar, PipeRead, PipeWrite);
// We no longer need the Write pipe so close it.
// We DO need the Read pipe however...
FPlatformProcess::ClosePipe(0, PipeWrite);
if (!ProcHandle.IsValid())
{
// We're done with the process handle now
ModuleCompileProcessHandle.Reset();
ModuleCompileReadPipe = NULL;
}
else
{
ModuleCompileProcessHandle = ProcHandle;
ModuleCompileReadPipe = PipeRead;
}
return ProcHandle.IsValid();
#else
return false;
#endif // WITH_HOT_RELOAD
}
void FHotReloadModule::CheckForFinishedModuleDLLCompile(const bool bWaitForCompletion, bool& bCompileStillInProgress, bool& bCompileSucceeded, FOutputDevice& Ar, bool bFireEvents)
{
#if WITH_HOT_RELOAD
bCompileStillInProgress = false;
ECompilationResult::Type CompilationResult = ECompilationResult::OtherCompilationError;
// Is there a compilation in progress?
if( !IsCurrentlyCompiling() )
{
Ar.Logf(TEXT("Error: CheckForFinishedModuleDLLCompile: There is no compilation in progress right now"));
return;
}
bCompileStillInProgress = true;
FText StatusUpdate;
if ( ModulesBeingCompiled.Num() > 0 )
{
FFormatNamedArguments Args;
Args.Add( TEXT("CodeModuleName"), FText::FromString( ModulesBeingCompiled[0].ModuleName ) );
StatusUpdate = FText::Format( NSLOCTEXT("FModuleManager", "CompileSpecificModuleStatusMessage", "{CodeModuleName}: Compiling modules..."), Args );
}
else
{
StatusUpdate = NSLOCTEXT("FModuleManager", "CompileStatusMessage", "Compiling modules...");
}
FScopedSlowTask SlowTask(0, StatusUpdate, GIsSlowTask);
SlowTask.MakeDialog();
// Check to see if the compile has finished yet
int32 ReturnCode = -1;
while (bCompileStillInProgress)
{
// Store the return code in a temp variable for now because it still gets overwritten
// when the process is running.
int32 ProcReturnCode = -1;
if( FPlatformProcess::GetProcReturnCode( ModuleCompileProcessHandle, &ProcReturnCode ) )
{
ReturnCode = ProcReturnCode;
bCompileStillInProgress = false;
}
if (bRequestCancelCompilation)
{
FPlatformProcess::TerminateProc(ModuleCompileProcessHandle);
bCompileStillInProgress = bRequestCancelCompilation = false;
}
if( bCompileStillInProgress )
{
ModuleCompileReadPipeText += FPlatformProcess::ReadPipe(ModuleCompileReadPipe);
if( !bWaitForCompletion )
{
// We haven't finished compiling, but we were asked to return immediately
break;
}
SlowTask.EnterProgressFrame(0.0f);
// Give up a small timeslice if we haven't finished recompiling yet
FPlatformProcess::Sleep( 0.01f );
}
}
bRequestCancelCompilation = false;
if( bCompileStillInProgress )
{
Ar.Logf(TEXT("Error: CheckForFinishedModuleDLLCompile: Compilation is still in progress"));
return;
}
// Compilation finished, now we need to grab all of the text from the output pipe
ModuleCompileReadPipeText += FPlatformProcess::ReadPipe(ModuleCompileReadPipe);
// This includes 'canceled' (-1) and 'up-to-date' (-2)
CompilationResult = (ECompilationResult::Type)ReturnCode;
// If compilation succeeded for all modules, go back to the modules and update their module file names
// in case we recompiled the modules to a new unique file name. This is needed so that when the module
// is reloaded after the recompile, we load the new DLL file name, not the old one.
// Note that we don't want to do anything in case the build was canceled or source code has not changed.
TMap<FString, FString> ChangedModules;
if(CompilationResult == ECompilationResult::Succeeded)
{
ChangedModules.Reserve(ModulesThatWereBeingRecompiled.Num());
for( FModuleToRecompile& CurModule : ModulesThatWereBeingRecompiled )
{
// Were we asked to assign a new file name for this module?
if( CurModule.NewModuleFilename.IsEmpty() )
{
continue;
}
if (IFileManager::Get().FileSize(*CurModule.NewModuleFilename) <= 0)
{
continue;
}
// If the file doesn't exist, then assume it doesn't needs rebinding because it wasn't recompiled
FDateTime FileTimeStamp = IFileManager::Get().GetTimeStamp(*CurModule.NewModuleFilename);
if (FileTimeStamp == FDateTime::MinValue())
{
continue;
}
FName ModuleName = *CurModule.ModuleName;
// If the file is the same as what we remembered it was then assume it doesn't needs rebinding because it wasn't recompiled
TSharedRef<FModuleCompilationData>* CompileDataPtr = ModuleCompileData.Find(ModuleName);
if (CompileDataPtr && (*CompileDataPtr)->FileTimeStamp == FileTimeStamp)
{
continue;
}
// If the compile succeeded, update the module info entry with the new file name for this module
OnModuleCompileSucceeded(ModuleName, CurModule.NewModuleFilename);
// Move modules
ChangedModules.Emplace(MoveTemp(CurModule.ModuleName), MoveTemp(CurModule.NewModuleFilename));
}
}
ModulesThatWereBeingRecompiled.Empty();
// We're done with the process handle now
FPlatformProcess::CloseProc(ModuleCompileProcessHandle);
ModuleCompileProcessHandle.Reset();
FPlatformProcess::ClosePipe(ModuleCompileReadPipe, 0);
Ar.Log(*ModuleCompileReadPipeText);
const FString FinalOutput = ModuleCompileReadPipeText;
ModuleCompileReadPipe = NULL;
ModuleCompileReadPipeText = TEXT("");
// No longer compiling modules
ModulesBeingCompiled.Empty();
bCompileSucceeded = !ECompilationResult::Failed(CompilationResult);
if ( bFireEvents )
{
const bool bShowLogOnSuccess = false;
ModuleCompilerFinishedEvent.Broadcast(FinalOutput, CompilationResult, !bCompileSucceeded || bShowLogOnSuccess);
// Fire task completion delegate
if (RecompileModulesCallback)
{
RecompileModulesCallback( ChangedModules, true, CompilationResult );
RecompileModulesCallback = nullptr;
}
}
#endif // WITH_HOT_RELOAD
}
void FHotReloadModule::UpdateModuleCompileData(FName ModuleName)
{
// Find or create a compile data object for this module
TSharedRef<FModuleCompilationData>* CompileDataPtr = ModuleCompileData.Find(ModuleName);
if(CompileDataPtr == nullptr)
{
CompileDataPtr = &ModuleCompileData.Add(ModuleName, TSharedRef<FModuleCompilationData>(new FModuleCompilationData()));
}
// reset the compile data before updating it
FModuleCompilationData& CompileData = CompileDataPtr->Get();
CompileData.bHasFileTimeStamp = false;
CompileData.FileTimeStamp = FDateTime(0);
CompileData.CompileMethod = EModuleCompileMethod::Unknown;
#if WITH_HOT_RELOAD
ReadModuleCompilationInfoFromConfig(ModuleName, CompileData);
FDateTime FileTimeStamp;
bool bGotFileTimeStamp = GetModuleFileTimeStamp(ModuleName, FileTimeStamp);
if (!bGotFileTimeStamp)
{
// File missing? Reset the cached timestamp and method to defaults and save them.
CompileData.bHasFileTimeStamp = false;
CompileData.FileTimeStamp = FDateTime(0);
CompileData.CompileMethod = EModuleCompileMethod::Unknown;
WriteModuleCompilationInfoToConfig(ModuleName, CompileData);
}
else
{
if (CompileData.bHasFileTimeStamp)
{
if (FileTimeStamp > CompileData.FileTimeStamp + HotReloadDefs::TimeStampEpsilon)
{
// The file is newer than the cached timestamp
// The file must have been compiled externally
CompileData.FileTimeStamp = FileTimeStamp;
CompileData.CompileMethod = EModuleCompileMethod::External;
WriteModuleCompilationInfoToConfig(ModuleName, CompileData);
}
}
else
{
// The cached timestamp and method are default value so this file has no history yet
// We can only set its timestamp and save
CompileData.bHasFileTimeStamp = true;
CompileData.FileTimeStamp = FileTimeStamp;
WriteModuleCompilationInfoToConfig(ModuleName, CompileData);
}
}
#endif
}
void FHotReloadModule::ReadModuleCompilationInfoFromConfig(FName ModuleName, FModuleCompilationData& CompileData)
{
FString DateTimeString;
if (GConfig->GetString(*HotReloadDefs::CompilationInfoConfigSection, *FString::Printf(TEXT("%s.TimeStamp"), *ModuleName.ToString()), DateTimeString, GEditorPerProjectIni))
{
FDateTime TimeStamp;
if (!DateTimeString.IsEmpty() && FDateTime::Parse(DateTimeString, TimeStamp))
{
CompileData.bHasFileTimeStamp = true;
CompileData.FileTimeStamp = TimeStamp;
FString CompileMethodString;
if (GConfig->GetString(*HotReloadDefs::CompilationInfoConfigSection, *FString::Printf(TEXT("%s.LastCompileMethod"), *ModuleName.ToString()), CompileMethodString, GEditorPerProjectIni))
{
if (CompileMethodString.Equals(HotReloadDefs::CompileMethodRuntime, ESearchCase::IgnoreCase))
{
CompileData.CompileMethod = EModuleCompileMethod::Runtime;
}
else if (CompileMethodString.Equals(HotReloadDefs::CompileMethodExternal, ESearchCase::IgnoreCase))
{
CompileData.CompileMethod = EModuleCompileMethod::External;
}
}
}
}
}
void FHotReloadModule::WriteModuleCompilationInfoToConfig(FName ModuleName, const FModuleCompilationData& CompileData)
{
FString DateTimeString;
if (CompileData.bHasFileTimeStamp)
{
DateTimeString = CompileData.FileTimeStamp.ToString();
}
GConfig->SetString(*HotReloadDefs::CompilationInfoConfigSection, *FString::Printf(TEXT("%s.TimeStamp"), *ModuleName.ToString()), *DateTimeString, GEditorPerProjectIni);
FString CompileMethodString = HotReloadDefs::CompileMethodUnknown;
if (CompileData.CompileMethod == EModuleCompileMethod::Runtime)
{
CompileMethodString = HotReloadDefs::CompileMethodRuntime;
}
else if (CompileData.CompileMethod == EModuleCompileMethod::External)
{
CompileMethodString = HotReloadDefs::CompileMethodExternal;
}
GConfig->SetString(*HotReloadDefs::CompilationInfoConfigSection, *FString::Printf(TEXT("%s.LastCompileMethod"), *ModuleName.ToString()), *CompileMethodString, GEditorPerProjectIni);
}
bool FHotReloadModule::GetModuleFileTimeStamp(FName ModuleName, FDateTime& OutFileTimeStamp) const
{
FString Filename = FModuleManager::Get().GetModuleFilename(ModuleName);
if (IFileManager::Get().FileSize(*Filename) > 0)
{
OutFileTimeStamp = FDateTime(IFileManager::Get().GetTimeStamp(*Filename));
return true;
}
return false;
}
bool FHotReloadModule::IsAnyGameModuleLoaded()
{
if (bIsAnyGameModuleLoaded == EThreeStateBool::Unknown)
{
bool bGameModuleFound = false;
// Ask the module manager for a list of currently-loaded gameplay modules
TArray< FModuleStatus > ModuleStatuses;
FModuleManager::Get().QueryModules(ModuleStatuses);
for (auto ModuleStatusIt = ModuleStatuses.CreateConstIterator(); ModuleStatusIt; ++ModuleStatusIt)
{
const FModuleStatus& ModuleStatus = *ModuleStatusIt;
// We only care about game modules that are currently loaded
if (ModuleStatus.bIsLoaded && ModuleStatus.bIsGameModule)
{
// There is at least one loaded game module.
bGameModuleFound = true;
break;
}
}
bIsAnyGameModuleLoaded = EThreeStateBool::FromBool(bGameModuleFound);
}
return EThreeStateBool::ToBool(bIsAnyGameModuleLoaded);
}
bool FHotReloadModule::ContainsOnlyGameModules(const TArray<FModuleToRecompile>& ModulesToCompile) const
{
const FString AbsoluteProjectDir(FPaths::ConvertRelativePathToFull(FPaths::ProjectDir()));
bool bOnlyGameModules = true;
for (auto& ModuleToCompile : ModulesToCompile)
{
const FString FullModulePath(FPaths::ConvertRelativePathToFull(ModuleToCompile.NewModuleFilename));
if (!FullModulePath.StartsWith(AbsoluteProjectDir))
{
bOnlyGameModules = false;
break;
}
}
return bOnlyGameModules;
}
void FHotReloadModule::ModulesChangedCallback(FName ModuleName, EModuleChangeReason ReasonForChange)
{
// Force update game modules state on the next call to IsAnyGameModuleLoaded
bIsAnyGameModuleLoaded = EThreeStateBool::Unknown;
// If the hot reload directory watcher hasn't been initialized yet (because the binaries directory did not exist) try to initialize it now
if (!bDirectoryWatcherInitialized)
{
RefreshHotReloadWatcher();
bDirectoryWatcherInitialized = true;
}
}
void FHotReloadModule::PluginMountedCallback(IPlugin& Plugin)
{
FDirectoryWatcherModule& DirectoryWatcherModule = FModuleManager::Get().LoadModuleChecked<FDirectoryWatcherModule>(TEXT("DirectoryWatcher"));
IDirectoryWatcher* DirectoryWatcher = DirectoryWatcherModule.Get();
if (DirectoryWatcher)
{
if (Plugin.GetLoadedFrom() == EPluginLoadedFrom::Project && Plugin.GetDescriptor().Modules.Num() > 0)
{
AddHotReloadDirectory(DirectoryWatcher, Plugin.GetBaseDir());
}
}
}
#undef LOCTEXT_NAMESPACE