Files
UnrealEngineUWP/Engine/Source/Programs/UnrealBuildTool/Linux/LinuxToolChain.cs
Nick Darnell 924baec97b Copying //UE4/Dev-Editor to //UE4/Dev-Main (Source: //UE4/Dev-Editor @ 3341527)
#lockdown Nick.Penwarden

==========================
MAJOR FEATURES + CHANGES
==========================

Change 3280282 on 2017/01/31 by Matt.Kuhlenschmidt

	GitHub 3171 : fix 'memoreport -full' causes ensure condition fail on particle object

Change 3281111 on 2017/02/01 by Michael.Dupuis

	#jira UE-36318 : was'nt notifying that we changed the current level in the case where you add/create new level in the Level window

Change 3281225 on 2017/02/01 by Jamie.Dale

	Several improvements to culture switching and LocRes files

	 - LocRes files now de-duplicate translations when they're generated, which can result in smaller LocRes files.
	 - The localization compilation step now produces a LocNat file, which contains meta-data specifying the native culture during compile, and where the native LocRes file can be found.
	 - Changing cultures now loads the native localization data prior to loading the non-native translations to ensure that translations are always applied to a consistent base.
	 - The "leet" culture (available when localization testing is enabled) is now always applied against the native translation, and correctly restores non-translated text when switching away from the "leet" culture.
	 - "-culture=leet" now works correctly on the command line ("-leet" also works).
	 - LoadLocalizationResourcesForCulture is no longer called multiple times during initialization of the text localization manager.
	 - General clean-up of localization code to favor using LocKeyFuncs with maps and sets, rather than rolling their own key funcs.

Change 3281291 on 2017/02/01 by Alexis.Matte

	Make sure the sections material slot assignation is persist correctly for staticmesh and for skeletal mesh
	#jira UE-39639

Change 3281718 on 2017/02/01 by Michael.Dupuis

	#jira UE-34186: invert processing order of special character, to take into account that key name could be considered a special character and would cause the assumption done to no longer be valid

Change 3281861 on 2017/02/01 by Alexis.Matte

	Fix import of morph target when there is no animation
	#jira UE-41383

Change 3282791 on 2017/02/02 by Chris.Wood

	Split crash analytics methods to fix comment parsing issues.
	[UE-32787] - Document Crash Report Client analytics events in code

Change 3283316 on 2017/02/02 by Alexis.Matte

	Make sure we do not import more then the maximum allowed node
	#jira UE-41405

Change 3283349 on 2017/02/02 by Jamie.Dale

	Updated Portal to stage its .locnat files

Change 3283927 on 2017/02/02 by Matt.Kuhlenschmidt

	Fix component/actor selection becoming out of sync after undo/redo

	#jira UE-41416

Change 3284061 on 2017/02/02 by Alexis.Matte

	Fix the scene importer front x axis import
	#jira UE-41318

Change 3284280 on 2017/02/02 by Alex.Delesky

	#jira UE-41060 - Placing blocking volumes in the level via the Content Menu's "Place Actor" command will now place a blocking volume in the level and not generate an empty warning in the output log

Change 3285053 on 2017/02/03 by Michael.Dupuis

	#jira UE-33777: Handle the global landscape editor ui command  list so specified shortcut will be treated

Change 3285444 on 2017/02/03 by Jamie.Dale

	Updated FastDecimalFormat to support the correct 0-9 numerals for the current locale

	These are typically still Latin, but Middle Eastern languages have some variants.

	This addresses an inconsistency between FText formatting of numbers and dates (since numbers always used Latin, but dates used the culture correct numerals).

Change 3287422 on 2017/02/06 by Michael.Dupuis

	#jira UE-36580: Improved the whole word algo to take into consideration localisation

Change 3287455 on 2017/02/06 by Alexis.Matte

	When swaping the mesh point by the mesh component, we noe clean up the override material instead of empty it.
	#jira UE-41397

Change 3287745 on 2017/02/06 by Alexis.Matte

	Merge from orion dev-general cl:3286668
	Fix a crash when importing a LOD containing different material with less sections

Change 3287996 on 2017/02/06 by Michael.Dupuis

	#jira UE-37290: fixed naming to be "move to level" instead of "move level"

Change 3288090 on 2017/02/06 by Jamie.Dale

	Fixing missing include breaking the FText natvis

Change 3288105 on 2017/02/06 by Jamie.Dale

	FTextStringHelper::ReadFromString_ComplexText now only looks at the start of the buffer when matching the complex text macros

Change 3288150 on 2017/02/06 by Jamie.Dale

	Fixing display names for tutorial categories so that they can be localized

	They were already FText, but the config wasn't defining them in a localizable way.

	#jira UE-37926

Change 3288469 on 2017/02/06 by Alex.Delesky

	#jira UE-35464 - Enables the editor to parse SubRip Subtitles files to create subtitle assets.

	This also introduces the Subtitles module.

Change 3288540 on 2017/02/06 by Alex.Delesky

	Backing out changelist 3288469 due to build issue with module includes

	#jira none

Change 3289074 on 2017/02/06 by Alex.Delesky

	Back out changelist 3288540 - reintroducing Subtitles module to parse SubRip Subtitles files

	#jira UE-35464

Change 3289753 on 2017/02/07 by Michael.Dupuis

	#jira UE-34599: Take into consideration UMaterialExpressionMaterialFunctionCall when getting the GUID

Change 3290097 on 2017/02/07 by Nick.Darnell

	Automation - The automation framework no longer buckets errors, warnings and log statements into a seperate set of buckets.  There is now only one log, and all entries go into it to provide some context when things fail.  Continued working on the styling of the reports.

Change 3290182 on 2017/02/07 by Michael.Trepka

	Added missing initialization for SWindow::bIsMirrorWindow

Change 3290472 on 2017/02/07 by Michael.Dupuis

	#jira UE-37358: Add reference list in the dialog for all delete type

Change 3290513 on 2017/02/07 by Michael.Dupuis

	#jira UE-37958: was testing the trailing number 0 twice and never testing the 1

Change 3290543 on 2017/02/07 by Michael.Dupuis

	#jira UE-35931: Refresh detail panel on selection lost

Change 3290581 on 2017/02/07 by Michael.Dupuis

	Fixed possible crash if we have no level blueprint specified (was crashing during the delete of an actor)

Change 3290721 on 2017/02/07 by Michael.Dupuis

	#jira UE-40360: Pass the custom spawning struct which contain the level override into to the spawn function

Change 3291958 on 2017/02/08 by Alexis.Matte

	Back out revision 26 from //UE4/Dev-Editor/Engine/Source/Developer/AssetTools/Private/AssetTools.cpp

Change 3292017 on 2017/02/08 by Alexis.Matte

	Add some fbx automation tests to validate material re-import

Change 3292030 on 2017/02/08 by Michael.Dupuis

	#jira UE-37958: was testing the trailing number 0 twice and never testing the 1

Change 3293062 on 2017/02/08 by Jamie.Dale

	Reduced the number of allocations that happen when rebuilding text

	This change removes the wasteful FTextHistory::ToText function and replaces it with two more specialized functions; FTextHistory::BuildLocalizedDisplayString and FTextHistory::BuildInvariantDisplayString.

	These new functions return an FString (for the display string), rather than an FText (which was simply mined for its display string). Simply avoiding going via an FText saves at least two allocations per-rebuild.

	Changes:
	 - Removed FTextHistory::ToText and replaced it with FTextHistory::BuildLocalizedDisplayString and FTextHistory::BuildInvariantDisplayString.
	 - Moved the localization aware chronological and transformation implementations into FTextChronoFormatter and FTextTransformer. These return an FString which avoids an FText allocation during rebuild, and is simply passed into an FText during normal FText usage.
	 - Moved FText::AsDate, FText::AsDateTime, FText::AsTime, FText::ToUpper, and FText::ToLower into Text.cpp, and these now use FTextChronoFormatter and FTextTransformer from the common text implementation.
	 - Moved FText::AsTimespan into Text.cpp. This had no dependency on ICU, so this is now the common text implementation.
	 - Added FTextFormatter::FormatStr variants. FTextFormatter::Format calls these FTextFormatter::FormatStr versions internally, and they're also used during text rebuilding (saving not only an FText allocation, but also a container copy).
	 - Removed FText::CreateNumericalText and FText::CreateChronologicalText as they were mostly superfluous.
	 - General update from using MakeShareable to MakeShared (saving 1 allocation).
	 - General clean-up of L10N/I18N class friendship.

	#jira UE-41533

Change 3293292 on 2017/02/08 by Alex.Delesky

	Performing some cleanup in the Subtitles module, and creating a SubtitlesEditor module for the subtitles asset factories since it causes issue in client builds.

Change 3293477 on 2017/02/08 by Jamie.Dale

	Fixed TProperty::InitializeValueInternal and TProperty::DestroyValueInternal mismatch when dealing with fixed size arrays

	#jira UE-41007

Change 3293571 on 2017/02/08 by Matt.Kuhlenschmidt

	Fix lots of outline data being added to the font cache due to wrongly hashing outline material and color data.

Change 3293572 on 2017/02/08 by Matt.Kuhlenschmidt

	Fix details panel categories in the static mesh editor

Change 3294216 on 2017/02/09 by Michael.Dupuis

	#jira UE-40609: manually position the window based on it'S max possible size
	#3128 GitHub

Change 3294430 on 2017/02/09 by Jamie.Dale

	Kerning-only text shaping no longer draws characters to get their metrics

	It now goes via the low-level FT caches like HarfBuzz does.

Change 3294588 on 2017/02/09 by Alexis.Matte

	If we remove a LODGroup from baseengine.ini, the fbx importer UI will now be able to recover in case the last fbx import was done with the just removed LODGroup

Change 3294847 on 2017/02/09 by Matt.Kuhlenschmidt

	Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)

Change 3295093 on 2017/02/09 by Arciel.Rekman

	Linux: fix Setup.sh not working in paths with space (UE-41819).

Change 3295205 on 2017/02/09 by Matt.Kuhlenschmidt

	Fix material UV's no longer working om 9 slice elements

Change 3295816 on 2017/02/09 by Arciel.Rekman

	Linux: fix starting programs from a path with space.

Change 3296129 on 2017/02/09 by Arciel.Rekman

	Linux i686: changes necessary to compile BlankProgram.

	- Added new architecture to UBT.
	- Fixed system headers.
	- Added third party libs for i686:
	  - jemalloc
	  - elftoolchain
	  - zlib
	  - SDL2
	  - libc++

Change 3296564 on 2017/02/10 by Jamie.Dale

	Cleaned up PO comment preservation

Change 3296694 on 2017/02/10 by Jamie.Dale

	AllocateNameEntry now takes TCharType* rather than void* and cast

Change 3296744 on 2017/02/10 by Jamie.Dale

	Moved the PO DOM from UnrealEd to Internationalization

Change 3297250 on 2017/02/10 by Jamie.Dale

	Split the PO import/export pipeline out of the commandlet

Change 3297420 on 2017/02/10 by Alexis.Matte

	Add Isolate and highlight feature for the material panel in the staticmesh and the skeletal editor.
	#jira UE-38985

Change 3297594 on 2017/02/10 by Alexis.Matte

	When importing from fbx a static mesh with find material anywhere, the next LODs import by the user will create new material entries instead of using the existing one.

Change 3297752 on 2017/02/10 by Arciel.Rekman

	i686 support: more third party libs.

	- libcurl
	- OpenSSL
	- libpng
	- libvorbis
	- libogg
	- libopus

Change 3297754 on 2017/02/10 by Arciel.Rekman

	i686 support: PhysX

Change 3297922 on 2017/02/10 by Alexis.Matte

	When importing a new LOD to a staticmesh, the data source file is not anymore wipe or change to the last fbx import filename.

Change 3298330 on 2017/02/10 by Arciel.Rekman

	i686: missing libcurl.

Change 3298620 on 2017/02/11 by Jamie.Dale

	FLocTextHelper improvements

	- It can now support non-standard target layouts (where the native and foreign cultures are in different locations - see FLocTextTargetPaths).
	- The XForeignArchive functions are now more strict, and *only* accept foreign cultures (use the XArchive functions instead if you're using both native and foreign cultures as parameters).

Change 3299293 on 2017/02/13 by Matt.Kuhlenschmidt

	PR #3241: UE-41870: Add quotes when passing through the directory path (Contributed by projectgheist)

Change 3299299 on 2017/02/13 by Matt.Kuhlenschmidt

	PR #3224: Git plugin: fix git autodetection and add error message (Contributed by SRombauts)

Change 3299391 on 2017/02/13 by Matt.Kuhlenschmidt

	Fix material instances being marked dirty when opening

	#jira UE-41721, UE-41719

Change 3299441 on 2017/02/13 by Nick.Darnell

	PR #3243: Fix bug that UWidget::GetOwningPlayer doesn't return (Contributed by yeonseok-yi)

Change 3299567 on 2017/02/13 by Nick.Darnell

	Slate - The Checkbox no longer just passes visibility down to the internal widgets it creates, that prevents future changes to effect it if it starts collapsed.

	#jira UE-41904

Change 3299870 on 2017/02/13 by Jamie.Dale

	Added cycle counters for font rendering/shaping

Change 3300116 on 2017/02/13 by Michael.Dupuis

	#jira UE-41866: Update cache when performing an undo

Change 3300178 on 2017/02/13 by Alexis.Matte

	Fix a crash when re-importing a LOD with more sections then the base LOD

Change 3300191 on 2017/02/13 by Alexis.Matte

	Make sure we do not loose castshadow and recomputetangents section flags when we re-import a skeletal mesh.

Change 3300351 on 2017/02/13 by Alexis.Matte

	Remove the clean up of unused material for the staticmesh editor. Unused material can be delete manually in the UI
	#jira UE-39639

Change 3302138 on 2017/02/14 by Nick.Darnell

	Automation - Adding support for -DeveloperReportOutputPath and -DeveloperReportUrl to permit local runs of the automation tool to generate reports on the report server, and launch the browser window to view them.

Change 3302139 on 2017/02/14 by Nick.Darnell

	UMG - Additional fixes to the way we migrate changes from the preview to the serialized version of the widget tree.  This fixes several issues with edit-inline objects on UWidgets.

Change 3302281 on 2017/02/14 by Nick.Darnell

	Slate - Bringing over changes to the invalidation panel from one of the game streams.  This fixes issues with animations in volatile widgets, as well as some issues with cache relative offset, and offers a method for enabling a different caching method to preserve batching through a commandline, but at the cost of not being able to use GPU buffers, possibly a better option on mobile in some cases.

Change 3302415 on 2017/02/14 by Nick.Darnell

	Disabling the open asset editor test.

Change 3302976 on 2017/02/14 by Nick.Darnell

	Automation - Updating one of the tests to open 70 different known asset types, and ensure that they open without dirtying the package.  AutomationTestSettings are now defaultengine, not sure why they setup to be user specific previously.  Most of these settings need to be removed, or split off into the modules that own them, rather than being in Engine.  TODO.

Change 3303724 on 2017/02/15 by Matt.Kuhlenschmidt

	Removed hard coded list of thumbnails, preventing objects with valid thumbnails from showing up.  Thumbnails are now shown by default.  Use meta=(DisplayThumbnail=false) to remove

	#jira UE-41958

Change 3303729 on 2017/02/15 by Matt.Kuhlenschmidt

	PR #3253: UE-34539: (Bugfix) Allow binary files in git stored via git-fat, git-lfs, etc to be diffed (take 2) (Contributed by rpav)

Change 3303733 on 2017/02/15 by Matt.Kuhlenschmidt

	PR #3248: Fix for TAssetSubClassOf properties reset on undo. (Contributed by StefanoProsperi)

Change 3303823 on 2017/02/15 by Nick.Darnell

	Automation - Continued improvements on screenshots.  Added some fixes to turn off the tonemapper when visualizing buffers.  Fixed several screenshots due to this change.  Adding lightboxes to the reports.  Adding some styling to make things sweeter.

Change 3303937 on 2017/02/15 by Matt.Kuhlenschmidt

	Fix build error

Change 3303982 on 2017/02/15 by Nick.Darnell

	Automation - Making the opening of the image no longer threaded, not really helpful for the IO operation and just makes it harder to follow.

Change 3304058 on 2017/02/15 by Matt.Kuhlenschmidt

	Fix build attempt #2 (not reproducible locally)

Change 3304393 on 2017/02/15 by Matt.Barnes

	Submitting test content for UEQATC-3548

Change 3304517 on 2017/02/15 by Nick.Darnell

	Slate - Making some fixes to the automatic disabling of the pixel snapping code with render transforms.  Sometimes it gets confused, we may want to move to a seperate transform stack for layout and render, and make sure the element drawer has access to both.

Change 3304560 on 2017/02/15 by Nick.Darnell

	UMG - SA fix.

Change 3304890 on 2017/02/15 by Matt.Kuhlenschmidt

	PR #3220: UE-41243: Force resolution in standalone if large than primary workin. (Contributed by projectgheist)

Change 3305360 on 2017/02/15 by Arciel.Rekman

	Linux: fix crash on exit (UE-41907).

	- It is not safe to dereference UAnimGraphNode_PoseDriver::StaticClass during the final shutdown sequence since the instance has already been destroyed in StaticExit().

Change 3306023 on 2017/02/16 by Nick.Darnell

	Paper2D - Adding a method to create SlateBrushes from PaperSprites the same way we can for materials and textures in blueprints.

Change 3306030 on 2017/02/16 by Nick.Darnell

	Slate - Making some additional fixes to invalidation panels from a game branch.  Adding a RoundToVector function to FVector2D, fixing the 3 places we defined a RoundToInt (which wasn't a great name since the convention wasn't meant to be used that way).

Change 3306031 on 2017/02/16 by Nick.Darnell

	Slate - Retainer widgets no longer tick using PreTick on SlateApplication, they now paint during their normal paint.

Change 3306046 on 2017/02/16 by Nick.Darnell

	UMG - Adding CanEditChange to WidgetComponent to gray out the CylinderArcAngle property unless you select the right geometry mode.

Change 3308887 on 2017/02/17 by Matt.Kuhlenschmidt

	Fix crash if blurs are rotated

	#jira UE-42037

Change 3309114 on 2017/02/17 by Jamie.Dale

	Unifying non-shaped text to use the same atlas cache as shaped text

Change 3310044 on 2017/02/17 by Matt.Kuhlenschmidt

	Outline color on text elements is now inherited properly

	#jira UE-40691

Change 3310268 on 2017/02/17 by Matt.Kuhlenschmidt

	Guard against rendering MIDs with potentially no parent in slate.

	#jira UE-42047

Change 3311531 on 2017/02/20 by Michael.Dupuis

	#jira UETOOL-1100:
	Add the possibility to have dynamic min/max slider value
	Synchonize all Color vector together when changing the min/max slider value

Change 3311534 on 2017/02/20 by Michael.Dupuis

	incremental build fix

Change 3311535 on 2017/02/20 by Michael.Dupuis

	incremental build fix take 2...

Change 3311743 on 2017/02/20 by Michael.Dupuis

	buildfix lunix incremental

Change 3312496 on 2017/02/20 by Arciel.Rekman

	Linux: fix PhysX crash in i686.

	- Changed layout to one that works.

Change 3313127 on 2017/02/20 by Jamie.Dale

	Fixed crash when performing a non-async cooked package save

	It isn't safe to call TotalSize on the BulkArchive when it's not a FBufferArchive (as used during async save) once the archive has been closed.

Change 3313990 on 2017/02/21 by Nick.Darnell

	Automation - Added a summary area at the top of the report.

Change 3314034 on 2017/02/21 by Jamie.Dale

	Fixed crash when deleting a streamed font

Change 3314942 on 2017/02/21 by Nick.Darnell

	Automation - More templating styling work.

Change 3315080 on 2017/02/21 by Nick.Darnell

	Automation - Providing a way for users to remove explict events from the event log when automated tests run.  Needed for other systems linked into the automation system like google mock.

Change 3315452 on 2017/02/21 by Nick.Darnell

	Json - Adding support for Map and Set properties to the JsonObjectConverter.  Can now save out map and sets.  No support for loading them yet.

Change 3315614 on 2017/02/21 by Nick.Darnell

	Json - Adding support for loading sets and map json data.

Change 3315924 on 2017/02/21 by Arciel.Rekman

	Vulkan: edigrating various Linux fixes by Josh.

	- This is to make Linux Vulkan work in Dev-Editor easier (for the contractor and myself).

	Original descriptions:

	CL 3313445
	- Various Vulkan fixes:
	  - Compiles in Linux
	  - Many cubemap bugs squashed
	  - Changed the scratch reflection cubemap clear to SetRenderTargestsAndClear, instead of SetRenderTarget() / Clear()
	  - Added compute fences

	CL 3314152
	- Fixed compile error on Mac, but I am pretty sure we can just remote VulkanRHI from Mac building entirely, but needs to be tested.

Change 3316741 on 2017/02/22 by Jamie.Dale

	Ensure that enums used by BP nodes have been PostLoaded so they have the correct display names

	#jira UE-42253

Change 3316800 on 2017/02/22 by Matt.Kuhlenschmidt

	Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)

Change 3317058 on 2017/02/22 by Alexis.Matte

	Fix the scene importer to support correctly the obj file format
	#jira UE-35606

Change 3318039 on 2017/02/22 by Arciel.Rekman

	i686 support: added missing libwebsockets.

Change 3318095 on 2017/02/22 by Arciel.Rekman

	i686 support: Oodle.

Change 3319002 on 2017/02/23 by Michael.Dupuis

	#jira UE-41794 : Do not exit the landscape mode when doing undo from the creation of the landscape

Change 3319012 on 2017/02/23 by Alexis.Matte

	PR #3066: Improve asset import by permitted relative paths and easing editing of mapped mount points. (Contributed by paulevans)
	#jira UE-40039

Change 3319035 on 2017/02/23 by Nick.Darnell

	UMG - Adding a note about the font sizes in UE4 in Slate, using 96 dpi.

	#jira UE-42170

Change 3319040 on 2017/02/23 by Matt.Kuhlenschmidt

	PR #3278: Git plugin: fix revision number for blueprint diff menu (Contributed by SRombauts)

	#jira UE-42129

Change 3319072 on 2017/02/23 by Michael.Dupuis

	#jira UETOOL-1101: Add support for DetailGroup reset to default
	Right now it's only enable for the color grading

Change 3319077 on 2017/02/23 by Nick.Darnell

	Automation - Moving away from most of the templating being done in C++.  Moving to dust.js to just do it in the browser window.  The json report file is now the actual source of the information we use to template the resulting report html.  Maaay have to move to doing the templating server side in the future to stream it to the client better, but avoiding that so we don't have to ship a server.  Disabling several places we were taking editor screenshots, none of that code was actually comparing screenshots, it was a hold-over from earlier days.

	PhysX - Fixing a problem with Physx FillInlinePxShapeArray.  Deprecating it, adding FillInlinePxShapeArray_AssumesLocked, and locking places we were assuming it was already locked in the landscape component.

Change 3319088 on 2017/02/23 by Nick.Darnell

	PR #3245: UE-41707: Re-order includes correctly (Contributed by projectgheist)

	#jira UE-41914

Change 3319104 on 2017/02/23 by Michael.Dupuis

	fix incremental build

Change 3319146 on 2017/02/23 by Matt.Kuhlenschmidt

	PR #3292: Git plugin: fix update status on directories broken since UE4.12 (Contributed by SRombauts)

	#jira UE-42272

Change 3319252 on 2017/02/23 by Michael.Dupuis

	fix warning with missing #undef LOCTEXT_NAMESPACE

Change 3319298 on 2017/02/23 by Alex.Delesky

	Removing the Subtitles and SubtitlesEditor modules (it'll eventually be brought back as the Overlay and OverlayEditor modules)

Change 3319388 on 2017/02/23 by Alexis.Matte

	Fbx Importer now find collision model under fbx LOD Group
	#jira UE-42141

Change 3319528 on 2017/02/23 by Michael.Dupuis

	Fixed Undo/Redo to be consistent with other vector modifcation behavior

Change 3319583 on 2017/02/23 by Alexis.Matte

	Fix the sample rate to use the least common multiplier of all keys
	#jira UE-42012

Change 3319705 on 2017/02/23 by Nick.Darnell

	Static Analysis - Fixing sonobjectconverter.cpp(460) : warning C6011: Dereferencing NULL pointer 'ArrayProperty'.

Change 3319711 on 2017/02/23 by Nick.Darnell

	Editor - Adding some checks to make sure the struct we're accessing is still a valid handle.

	#jira UE-42262

Change 3319736 on 2017/02/23 by Alex.Delesky

	Adding Subtitles and SubtitlesEditor to the JunkManifest file.

Change 3319919 on 2017/02/23 by Nick.Darnell

	Automation - Fixing an issue with moving a location that doesn't exist.

Change 3319932 on 2017/02/23 by Alexis.Matte

	Fbx importer, do not apply more then one time the transform option to the scene node.
	#jira UE-42277

Change 3320105 on 2017/02/23 by Nick.Darnell

	Editor - Adding some additional checks to the margin customization.

	#jira UE-42262

Change 3321577 on 2017/02/24 by Jamie.Dale

	Moving Internationalization module from Runtime to Developer

Change 3321625 on 2017/02/24 by Jamie.Dale

	Moving InternationalizationSettings module from Developer to Editor

Change 3321642 on 2017/02/24 by Jamie.Dale

	Moving SCulturePicker from the Localization module to the InternationalizationSettings module

Change 3321734 on 2017/02/24 by Alexis.Matte

	PR #2979: Fix extra root bone for Blender exported FBX. (Contributed by manmohanbishnoi)
	We fix the extra root only when the file creator is from blender and the root node is named armature. We cannot simply remove all dummy node, since this is use by the rigid mesh workflow.

	#jira UE-39050

Change 3321912 on 2017/02/24 by Jamie.Dale

	Split LocalizationCommandletExecution out of the Localization module to remove some editor dependencies

Change 3322274 on 2017/02/24 by Jamie.Dale

	Moving Localization module from Editor to Developer, and merging the Internationalization module into it

	Removed hard-dependency between Engine and Localization/Internationalization via an interface.

Change 3322774 on 2017/02/25 by Jamie.Dale

	Unifying LocRes and LocNat file format between generation and loading

	This lets the code in Core be shared by Localization, and allows some code that was proxying via archives (due to the code being logically identical, but different C++ types) to use these new types directly.

	#tests Built Debug, Shipping, and Editor. Verified that LocNat and LocRes generation and loading worked as before.

Change 3322795 on 2017/02/25 by Jamie.Dale

	Fixing mismatch between SOURCE_CONTROL_WITH_SLATE and its .Build.cs file

	The define was set to disable Slate for Linux program targets only, but the .Build.cs disabled Slate for all Linux targets.

	Since the define was touched most recently (CL# 2534983), I updated the .Build.cs file to match its logic, and moved the definition of the define to the .Build.cs file so that they stay in sync with one another.

Change 3322853 on 2017/02/25 by Jamie.Dale

	Moved the conflict and word count reporting to FLocTextHelper

Change 3323089 on 2017/02/26 by Jamie.Dale

	Added functions to get the target name and path from FLocTextHelper

Change 3323391 on 2017/02/27 by Ben.Cosh

	This fixes an issue with blueprint config variables having their value destroyed by CDO serialization
	#Jira UE-40586 Blueprint variable defaults set from config files value are overwritten by CDO serialization
	#Proj Engine, CoreUObject

Change 3323406 on 2017/02/27 by Ben.Cosh

	Fixed a problem that caused UK2Node::ExpandSplitPin to destroy pins it didn't own in when expanding a collapsed graph during compilation.
	#jira UE-41211 - Crash when splitting a UDS pin on a collapsed graph
	#Proj BlueprintGraph

Change 3323572 on 2017/02/27 by Nick.Darnell

	Automation - Continued itteration on the style of the automation reports, now with attentional info, like where the log came from.

	Automation - Fixing a bug in the functional actor tests, navigating to the actors sometimes opened other objects in the package, now it only opens the map.  Also improved the way we focus the actor so that the level editor is also brought to the foreground.

	Automation - Fixing a bug in how the automation system was registering for capturing logging.  It was swapping out GWarn for its own version, but GWarn isn't called for anything that isn't an error or warning, meaning that none of the Display/Logging or analytics capture attempts were actually working.  Suddenly a flood of informations started being captured during tests.  For now - only going to capture 'Display' logs instead of 'Log' level.

	Automation - Successful comparisons now print more information so that the automation logs do a better job of tracking the flow of the test.

	Automation - The screenshot comparison test now prints more information even during successful comparisons.

	Editor - The message log no longer emits a SetSelection, just because the selection is updated the categoriry view model.  This was causing things like the automation tool, which sets the selection every time (which may itself be an issue) to completely rebuild the message log every time a new automation message was emited.  The message log now checks if the selection would actually change the viewstate before it does it.

	Domino Test - Adding an arrow to visualize the state of the up vector the test is looking for; playing with idea for test visualizers that may help with debugging in the future.

Change 3323580 on 2017/02/27 by Michael.Trepka

	Fixed some Xcode 8.3 compile errors

Change 3323634 on 2017/02/27 by Nick.Darnell

	Build - Fix incremental build.

Change 3323740 on 2017/02/27 by Jamie.Dale

	Adding #error if the SOURCE_CONTROL_WITH_SLATE define is missing

Change 3323865 on 2017/02/27 by Nick.Darnell

	Automation - Disabling the screenshot from the small editor icons test, until the editor screenshot method starts comparing things, and the screenshots we take are better / more scoped.

Change 3324228 on 2017/02/27 by Jamie.Dale

	Can no longer name assets or folders with a leading underscore

	#jira UE-40541

Change 3324429 on 2017/02/27 by Jamie.Dale

	Removing FLocTextTargetPaths

	It was added to support something that I'm now going to do a different way.

Change 3324473 on 2017/02/27 by Jamie.Dale

	Moved the GatherText SCC utils into the Localization module

Change 3324481 on 2017/02/27 by Jamie.Dale

	Moving the localized asset utils out of GatherText base

Change 3324485 on 2017/02/27 by Jamie.Dale

	Cleaning up some includes now that the localization SCC is no longer in GatherText

Change 3324910 on 2017/02/28 by Nick.Darnell

	Slate - Moving the SlateRotatedRect into its own file, and removing FSlateRotatedClipRectType, since there's no longer a difference and we only use FSlateRotatedRect.

Change 3325329 on 2017/02/28 by Michael.Dupuis

	#jira UE-42083: Removed various Modify(true) that would force user to save the levels even if they did'nt really modified them
	Replace TMap<TLazyObjectPtr,...> as it would dirty the level at every Find performed

Change 3325410 on 2017/02/28 by Michael.Dupuis

	missing include for incremental build

Change 3325415 on 2017/02/28 by Nick.Darnell

	UMG - Adding some setters and getters for RedrawTime to the WidgetComponent.

Change 3325418 on 2017/02/28 by Nick.Darnell

	Automation  - Fixing the warnings on startup about smoke tests taking longer than 2s.  Had to add an option to disable capturing the callstack when running smokes, it adds a bit too much overhead during startup.

Change 3325698 on 2017/02/28 by Alexis.Matte

	Put back the code to isolate material versus section in the skeletal mesh. The code was override by a temporary hack done in paragon branch

Change 3325790 on 2017/02/28 by Michael.Trepka

	Copy of CL 3319588

	Fixed address sanitizer support in MacToolChain (Apple changed the name of the env variable Xcode uses to enable it) and added support for thread sanitizer

Change 3326118 on 2017/02/28 by Alexis.Matte

	Add LOD settings LOD distances to fbx import dialog option. The option are not supported yet by the scene importer
	#jira UE-41291

Change 3326183 on 2017/02/28 by Alexis.Matte

	PR #3298: Import SpecularFactor for Roughness and Shininess for Metallic textures (Contributed by VladimirPobedinskiy)

	#jira UE-42301

Change 3326196 on 2017/02/28 by Jamie.Dale

	Force the correct package localization ID when duplicating a BP for nativization

Change 3327037 on 2017/03/01 by Michael.Dupuis

	fixed fortnite mac non editor build

Change 3327483 on 2017/03/01 by Jamie.Dale

	Renaming LocNat to LocMeta

Change 3327486 on 2017/03/01 by Jamie.Dale

	Renaming LocNat to LocMeta

Change 3327541 on 2017/03/01 by Michael.Trepka

	Removed Mac OpenGL RHI files and disabled building of OpenGL RHI on Mac

Change 3328000 on 2017/03/01 by Nick.Darnell

	Automation - Noisy rendering features are now disabled by default when taking screenshots.

Change 3328323 on 2017/03/01 by Michael.Trepka

	Copy of CL 3307526

	Fixed mouse position issues in fullscreen mode on Mac

Change 3328410 on 2017/03/01 by Alexis.Matte

	Remove unwanted option when importing skeletal mesh
	Make the FBX tests uptodate with the new ImportUI options

	#jira UE-41291

Change 3329586 on 2017/03/02 by Jamie.Dale

	Adding missing includes when running with bUseMallocProfiler enabled

Change 3329999 on 2017/03/02 by Nick.Darnell

	UMG - Removing a deprecated 4.8 function to get the label on UWidget.

Change 3330004 on 2017/03/02 by Nick.Darnell

	UMG - Adding TargetPlatform to the dependencies of UMGEditor module.

Change 3330021 on 2017/03/02 by Nick.Darnell

	UMG - Adding TargetPlatform to the private include path of the UMG module.

Change 3330041 on 2017/03/02 by Nick.Darnell

	Engine - Adding a comment to the PreLoadMap call so people know what the string being passed in is.

Change 3330048 on 2017/03/02 by Nick.Darnell

	Editor - Don't allow querying the cursor in the editor viewport while saving packages.  Depending upon the code that gets triggered, it may cause packages to load, or things to be initialized while saving is occuring.

Change 3330602 on 2017/03/02 by mason.seay

	Map for Functional Screenshot Test Bug

Change 3330632 on 2017/03/02 by Alexis.Matte

	Fix fbx crash when there is only one UVChannel but using the naming convention to place it further then the first index

Change 3330862 on 2017/03/02 by Jamie.Dale

	Adding FPaths::SetExtension

	This is like FPaths::ChangeExtension, but also applies the extension if the file doesn't have one.

Change 3331491 on 2017/03/03 by Nick.Darnell

	Automation - Fixing a threading issue in the SAsyncImage, it was accessing potentially bogus memory if the Widget had been deleted before the task ran.

Change 3331498 on 2017/03/03 by Nick.Darnell

	Build - Fixing a build warning.

Change 3331807 on 2017/03/03 by Nick.Darnell

	Automation - Making the Disable Noisy Rendering Features more robust, disabling a few more markers.  Adding a better way of rolling back the changes.

Change 3331999 on 2017/03/03 by Michael.Trepka

	Fixed a memory leak on texture creation with BulkData in OpenGLTexture.cpp

Change 3332481 on 2017/03/03 by Arciel.Rekman

	Fix building lighting in commandlet (UE-42551).

	- Process task graph while running as commandlet.
	- Also, if for any reason - like the lack of -messaging - local swarm interface fails to initialize or takes too much time to send the message, bail out.

Change 3332606 on 2017/03/04 by Jamie.Dale

	Fixing crash reporting loc word counts when the report is starting empty

Change 3332614 on 2017/03/04 by Jamie.Dale

	Fixed text namespaces being treated as case-insensitive when export to JSON manifests and archives

Change 3332619 on 2017/03/04 by Jamie.Dale

	Fixing CIS error

Change 3333000 on 2017/03/06 by Matt.Kuhlenschmidt

	PR #3295: Non-editable FStringAssetReference using VisibleAnywhere (Contributed by projectgheist)

	#jira UE-42284

Change 3333039 on 2017/03/06 by Alexis.Matte

	Make custom ui for FbxSceneImportData object
	#jira UE-37896

Change 3333047 on 2017/03/06 by Nick.Darnell

	UMG - Removing an extra assignment in WidgetSwitcher.

Change 3333056 on 2017/03/06 by Alexis.Matte

	Build fix

Change 3333073 on 2017/03/06 by Matt.Kuhlenschmidt

	Added more logging for when window creation fails due to too many windows.

	#jira UE-42478

Change 3333081 on 2017/03/06 by Matt.Kuhlenschmidt

	PR #3327: Git Plugin: fix RunDumpToFile() to check git ReturnCode (Contributed by SRombauts)

	#jira UE-42535

Change 3333103 on 2017/03/06 by Matt.Kuhlenschmidt

	PR #3336: UE-42407: using GetWindowMode instead of switching on IsFullscreenViewport (Contributed by stefanzimecki)

	#jira UE-42407, UE-42565

Change 3333142 on 2017/03/06 by Jamie.Dale

	Added a way to view/copy a list references (including those that aren't loaded) to the reference viewer

Change 3333443 on 2017/03/06 by Matt.Kuhlenschmidt

	Eliminate the usage of SWebBrowser to show viewport controls in level viewports. There is an non-trivial startup cost initializing CEF and is not worth paying that cost on editor startup for one tiny control.  The button now opens a web page on click.

	#jira UE-42461
	PR #3314: Drop UE4Editor -> CEF dependency to 2x speedup Linux UE4Editor startup (Contributed by slonopotamus)

Change 3333914 on 2017/03/06 by Matt.Kuhlenschmidt

	Remove double middle mouse click to change to perspective view

	#jira UE-42444

Change 3333936 on 2017/03/06 by Matt.Kuhlenschmidt

	Fixed excessive fname initialization in these files

Change 3334063 on 2017/03/06 by Alexis.Matte

	fix build linux

Change 3334166 on 2017/03/06 by Jamie.Dale

	Adding Data Table export/import support for TMap and TSet

	#jira UE-42415

Change 3334459 on 2017/03/06 by Alexis.Matte

	PR #3334: Respect bForceFrontXAxis option when exporting to FBX (Contributed by rajkosto)

	#jira UE-42563

Change 3335132 on 2017/03/07 by Jamie.Dale

	Fixing typo

Change 3335140 on 2017/03/07 by Jamie.Dale

	Fixing CSV import warnings in GameplayEffects test

Change 3335164 on 2017/03/07 by Alexis.Matte

	Avoid selecting skeletal mesh section in the level when high light them in persona editor
	#jira UE-20151

Change 3335186 on 2017/03/07 by Jamie.Dale

	Fixed CSV parser missing empty cells at the end of the string

Change 3335218 on 2017/03/07 by Arciel.Rekman

	SDL2: delete unused project/build files.

Change 3335222 on 2017/03/07 by Arciel.Rekman

	SDL2: delete more unused project/build files.

Change 3335230 on 2017/03/07 by Matt.Kuhlenschmidt

	Additional fixes for blur and blur slot not propagating padding to each other

	#jira UE-42553

Change 3335896 on 2017/03/07 by Jamie.Dale

	ToolTips and Engine were double gathering the same meta-data

	#jira UE-36480

Change 3336009 on 2017/03/07 by Matt.Kuhlenschmidt

	Fix details panels becoming unusable if "Show only Modified Properties" is enabled and there are no modified properties

Change 3336247 on 2017/03/07 by Jamie.Dale

	Selection height is now the max of the line height and text height to account for negative line scaling

	#jira UE-40673

Change 3336253 on 2017/03/07 by Jamie.Dale

	Added a setting to control whether we should use the font metrics or the bounding box when laying out a font

	#jira UE-41074

Change 3336303 on 2017/03/07 by Arciel.Rekman

	Refactor of OS memory allocation functions.

	- Bring PageSize/OSAllocationGranularity in line with the established definitions.
	  - PageSize is a hardware mapping granularity that is also used for PageProtect() and any other functions that involve setting virtual memory properties.
	  - OSAllocationGranularity is a virtual address allocation granularity that on some platforms may be applied on top of that (notably VirtualAlloc in Windows only returns addresses that are 16 page aligned).
	  - BinnedPageSize and BinnedAllocationGranularity are the values expected by Binned and Binned2 for size and the alignment of OS allocations.

	- Disable the logic in CachedOSPageAllocator that allowed buffers larger than the requested size to be returned.
	   - This caused wrong allocation size to be passed in BinnedFreeToOS() from Binned2.

	- Make Binned2 work on Linux
	    - Addresses returned from BinnedAllocFromOS() need to be BinnedPageSize (minimum 64KB) aligned for Binned2 to work. This results in the need to artificially align mmap()'d addresses, at some performance cost.
	    - The same function can be used on other systems with mmap()/munmap() (Mac, Android, iOS)

	- Switch Linux to Binned2 by default.

	- Add ability to sanity-check OS memory allocations.
	   - Debug and Development build will store a descriptor to check that values passed to BinnedFreeToOS() are the same (mmap-based allocation only).

Change 3337098 on 2017/03/08 by Michael.Dupuis

	#jira UE-42589: Added a guard if the mesh component is not attached, this can happen when moving a component out of the screen

Change 3337183 on 2017/03/08 by Matt.Kuhlenschmidt

	Hide the preview toolbar button, it is not being used

Change 3337801 on 2017/03/08 by Michael.Trepka

	Fixed some module dependencies to make sure we don't build OpenGLDrv on Mac

Change 3338373 on 2017/03/08 by Joe.Graf

	Fixed external plugin cooking and deployment by remapping plugin directories upon cook & deployment
	Tested directory structures:
	    D:\SomePluginDir
	    D:\UE4\AnotherPluginDir
	    D:\UE4\Engine\Plugins
	    D:\UE4\MyProject\Plugins

Change 3338482 on 2017/03/08 by Alexis.Matte

	Remove "BlueprinReadOnly" flag on "WITH_EDITORONLY_DATA" class variable

Change 3338679 on 2017/03/08 by Matt.Kuhlenschmidt

	Fixed arrow keys not working to navigate between elements in the details panel

Change 3339086 on 2017/03/09 by Dmitriy.Dyomin

	Added: Mobile friendly slate settings

Change 3339366 on 2017/03/09 by Nick.Darnell

	Build - Attempting to fix build.

	#jira UE-42675

Change 3339506 on 2017/03/09 by Jamie.Dale

	Fixing Linux Server build error

	#jira UE-42675

Change 3340450 on 2017/03/09 by Cody.Albert

	Ensure that the hittest grid is valid before trying to find a focusable widget

Change 3340492 on 2017/03/09 by Arciel.Rekman

	Fix IOS compile error (UE-42695).

Change 3340565 on 2017/03/09 by Arciel.Rekman

	Fix another compile error (UE-42695).

Change 3341527 on 2017/03/10 by Alexis.Matte

	Fix crash when dragging a re-import scene and there is new asset created
	#jira UE-42766

[CL 3341914 by Nick Darnell in Main branch]
2017-03-10 15:37:02 -05:00

1371 lines
50 KiB
C#

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.
using System;
using System.Collections.Generic;
using System.Text.RegularExpressions;
using System.Diagnostics;
using System.IO;
using System.Linq;
using Microsoft.Win32;
namespace UnrealBuildTool
{
class LinuxToolChain : UEToolChain
{
string Architecture;
public LinuxToolChain(string InArchitecture)
: base(CppPlatform.Linux)
{
Architecture = InArchitecture;
if (!CrossCompiling())
{
// use native linux toolchain
string[] ClangNames = { "clang++", "clang++-3.9", "clang++-3.8", "clang++-3.7", "clang++-3.6", "clang++-3.5" };
foreach (var ClangName in ClangNames)
{
ClangPath = Which(ClangName);
if (!String.IsNullOrEmpty(ClangPath))
{
break;
}
}
GCCPath = Which("g++");
ArPath = Which("ar");
RanlibPath = Which("ranlib");
StripPath = Which("strip");
// if clang is available, zero out gcc (@todo: support runtime switching?)
if (!String.IsNullOrEmpty(ClangPath))
{
GCCPath = null;
}
}
else
{
// if new multi-arch toolchain is used, prefer it
string MultiArchRoot = Environment.GetEnvironmentVariable("LINUX_MULTIARCH_ROOT");
if (MultiArchRoot != null)
{
// FIXME: UBT should loop across all the architectures and compile for all the selected ones.
BaseLinuxPath = Path.Combine(MultiArchRoot, Architecture);
Console.WriteLine("Using LINUX_MULTIARCH_ROOT, building with toolchain '{0}'", BaseLinuxPath);
}
else
{
// use cross linux toolchain if LINUX_ROOT is specified
BaseLinuxPath = Environment.GetEnvironmentVariable("LINUX_ROOT");
Console.WriteLine("Using LINUX_ROOT (deprecated, consider LINUX_MULTIARCH_ROOT), building with toolchain '{0}'", BaseLinuxPath);
}
// don't register if we don't have an LINUX_ROOT specified
if (String.IsNullOrEmpty(BaseLinuxPath))
{
throw new BuildException("LINUX_ROOT environment variable is not set; cannot instantiate Linux toolchain");
}
BaseLinuxPath = BaseLinuxPath.Replace("\"", "");
// set up the path to our toolchains
GCCPath = "";
ClangPath = Path.Combine(BaseLinuxPath, @"bin\clang++.exe");
// ar and ranlib will be switched later to match the architecture
ArPath = "ar.exe";
RanlibPath = "ranlib.exe";
StripPath = "strip.exe";
}
if (!DetermineCompilerVersion())
{
throw new BuildException("Could not determine version of the compiler, not registering Linux toolchain.");
}
// refuse to use compilers that we know won't work
// disable that only if you are a dev and you know what you are doing
if (!UsingClang())
{
throw new BuildException("This version of the engine can only be compiled by clang - refusing to register the Linux toolchain.");
}
else if (CompilerVersionMajor == 3 && CompilerVersionMinor == 4)
{
throw new BuildException("clang 3.4.x is known to miscompile the engine - refusing to register the Linux toolchain.");
}
// prevent unknown clangs since the build is likely to fail on too old or too new compilers
else if ((CompilerVersionMajor * 10 + CompilerVersionMinor) > 39 || (CompilerVersionMajor * 10 + CompilerVersionMinor) < 35)
{
throw new BuildException(
string.Format("This version of the Unreal Engine can only be compiled with clang 3.9, 3.8, 3.7, 3.6 and 3.5. clang {0} may not build it - please use a different version.",
CompilerVersionString)
);
}
}
protected static bool CrossCompiling()
{
return BuildHostPlatform.Current.Platform != UnrealTargetPlatform.Linux;
}
protected static bool UsingClang()
{
return !String.IsNullOrEmpty(ClangPath);
}
private string Which(string name)
{
Process proc = new Process();
proc.StartInfo.FileName = "/bin/sh";
proc.StartInfo.Arguments = String.Format("-c 'which {0}'", name);
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.CreateNoWindow = true;
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.RedirectStandardError = true;
proc.Start();
proc.WaitForExit();
string path = proc.StandardOutput.ReadLine();
Log.TraceVerbose(String.Format("which {0} result: ({1}) {2}", name, proc.ExitCode, path));
if (proc.ExitCode == 0 && String.IsNullOrEmpty(proc.StandardError.ReadToEnd()))
{
return path;
}
return null;
}
/// <summary>
/// Splits compiler version string into numerical components, leaving unchanged if not known
/// </summary>
private void DetermineCompilerMajMinPatchFromVersionString()
{
string[] Parts = CompilerVersionString.Split('.');
if (Parts.Length >= 1)
{
CompilerVersionMajor = Convert.ToInt32(Parts[0]);
}
if (Parts.Length >= 2)
{
CompilerVersionMinor = Convert.ToInt32(Parts[1]);
}
if (Parts.Length >= 3)
{
CompilerVersionPatch = Convert.ToInt32(Parts[2]);
}
}
/// <summary>
/// Queries compiler for the version
/// </summary>
private bool DetermineCompilerVersion()
{
CompilerVersionString = null;
CompilerVersionMajor = -1;
CompilerVersionMinor = -1;
CompilerVersionPatch = -1;
Process Proc = new Process();
Proc.StartInfo.UseShellExecute = false;
Proc.StartInfo.CreateNoWindow = true;
Proc.StartInfo.RedirectStandardOutput = true;
Proc.StartInfo.RedirectStandardError = true;
if (!String.IsNullOrEmpty(GCCPath) && File.Exists(GCCPath))
{
Proc.StartInfo.FileName = GCCPath;
Proc.StartInfo.Arguments = " -dumpversion";
Proc.Start();
Proc.WaitForExit();
if (Proc.ExitCode == 0)
{
// read just the first string
CompilerVersionString = Proc.StandardOutput.ReadLine();
DetermineCompilerMajMinPatchFromVersionString();
}
}
else if (!String.IsNullOrEmpty(ClangPath) && File.Exists(ClangPath))
{
Proc.StartInfo.FileName = ClangPath;
Proc.StartInfo.Arguments = " --version";
Proc.Start();
Proc.WaitForExit();
if (Proc.ExitCode == 0)
{
// read just the first string
string VersionString = Proc.StandardOutput.ReadLine();
Regex VersionPattern = new Regex("version \\d+(\\.\\d+)+");
Match VersionMatch = VersionPattern.Match(VersionString);
// version match will be like "version 3.3", so remove the "version"
if (VersionMatch.Value.StartsWith("version "))
{
CompilerVersionString = VersionMatch.Value.Replace("version ", "");
DetermineCompilerMajMinPatchFromVersionString();
}
}
}
else
{
// icl?
}
if (!CrossCompiling() && !ProjectFileGenerator.bGenerateProjectFiles)
{
Console.WriteLine("Using {0} version '{1}' (string), {2} (major), {3} (minor), {4} (patch)",
String.IsNullOrEmpty(ClangPath) ? "gcc" : "clang",
CompilerVersionString, CompilerVersionMajor, CompilerVersionMinor, CompilerVersionPatch);
}
return !String.IsNullOrEmpty(CompilerVersionString);
}
/// <summary>
/// Checks if compiler version matches the requirements
/// </summary>
private static bool CompilerVersionGreaterOrEqual(int Major, int Minor, int Patch)
{
return CompilerVersionMajor > Major ||
(CompilerVersionMajor == Major && CompilerVersionMinor > Minor) ||
(CompilerVersionMajor == Major && CompilerVersionMinor == Minor && CompilerVersionPatch >= Patch);
}
/// <summary>
/// Architecture-specific compiler switches
/// </summary>
static string ArchitectureSpecificSwitches(string Architecture)
{
string Result = "";
if (Architecture.StartsWith("arm") || Architecture.StartsWith("aarch64"))
{
Result += " -fsigned-char";
}
return Result;
}
static string ArchitectureSpecificDefines(string Architecture)
{
string Result = "";
if (Architecture.StartsWith("x86_64") || Architecture.StartsWith("aarch64"))
{
Result += " -D_LINUX64";
}
return Result;
}
/// <summary>
/// Gets architecture-specific ar paths
/// </summary>
private static string GetArPath(string Architecture)
{
if (CrossCompiling())
{
return Path.Combine(Path.Combine(BaseLinuxPath, String.Format("bin/{0}-{1}", Architecture, ArPath)));
}
return ArPath;
}
/// <summary>
/// Gets architecture-specific ranlib paths
/// </summary>
private static string GetRanlibPath(string Architecture)
{
if (CrossCompiling())
{
return Path.Combine(Path.Combine(BaseLinuxPath, String.Format("bin/{0}-{1}", Architecture, RanlibPath)));
}
return RanlibPath;
}
/// <summary>
/// Gets architecture-specific strip path
/// </summary>
private static string GetStripPath(string Architecture)
{
if (CrossCompiling())
{
return Path.Combine(Path.Combine(BaseLinuxPath, String.Format("bin/{0}-{1}", Architecture, StripPath)));
}
return StripPath;
}
private static bool ShouldUseLibcxx(string Architecture)
{
// set UE4_LINUX_USE_LIBCXX to either 0 or 1. If unset, defaults to 1.
string UseLibcxxEnvVarOverride = Environment.GetEnvironmentVariable("UE4_LINUX_USE_LIBCXX");
if (UseLibcxxEnvVarOverride != null && (UseLibcxxEnvVarOverride == "1"))
{
return true;
}
// at the moment ARM32 libc++ remains missing
return Architecture.StartsWith("x86_64") || Architecture.StartsWith("aarch64") || Architecture.StartsWith("i686");
}
static string GetCLArguments_Global(CppCompileEnvironment CompileEnvironment)
{
string Result = "";
// build up the commandline common to C and C++
Result += " -c";
Result += " -pipe";
if (ShouldUseLibcxx(CompileEnvironment.Architecture))
{
Result += " -nostdinc++";
Result += " -I" + "ThirdParty/Linux/LibCxx/include/";
Result += " -I" + "ThirdParty/Linux/LibCxx/include/c++/v1";
}
Result += " -Wall -Werror";
// test without this next line?
Result += " -funwind-tables"; // generate unwind tables as they seem to be needed for stack tracing (why??)
Result += " -Wsequence-point"; // additional warning not normally included in Wall: warns if order of operations is ambigious
//Result += " -Wunreachable-code"; // additional warning not normally included in Wall: warns if there is code that will never be executed - not helpful due to bIsGCC and similar
//Result += " -Wshadow"; // additional warning not normally included in Wall: warns if there variable/typedef shadows some other variable - not helpful because we have gobs of code that shadows variables
Result += ArchitectureSpecificSwitches(CompileEnvironment.Architecture);
Result += " -fno-math-errno"; // do not assume that math ops have side effects
Result += GetRTTIFlag(CompileEnvironment); // flag for run-time type info
if (String.IsNullOrEmpty(ClangPath))
{
// GCC only option
Result += " -fno-strict-aliasing";
Result += " -Wno-sign-compare"; // needed to suppress: comparison between signed and unsigned integer expressions
Result += " -Wno-enum-compare"; // Stats2.h triggers this (ALIGNOF(int64) <= DATA_ALIGN)
Result += " -Wno-return-type"; // Variant.h triggers this
Result += " -Wno-unused-local-typedefs";
Result += " -Wno-multichar";
Result += " -Wno-unused-but-set-variable";
Result += " -Wno-strict-overflow"; // Array.h:518
}
else
{
// Clang only options
if (CrossCompiling())
{
Result += " -fdiagnostics-format=msvc"; // make diagnostics compatible with MSVC when cross-compiling
}
Result += " -Wno-unused-private-field"; // MultichannelTcpSocket.h triggers this, possibly more
// this hides the "warning : comparison of unsigned expression < 0 is always false" type warnings due to constant comparisons, which are possible with template arguments
Result += " -Wno-tautological-compare";
// this switch is understood by clang 3.5.0, but not clang-3.5 as packaged by Ubuntu 14.04 atm
if (CompilerVersionGreaterOrEqual(3, 5, 0))
{
Result += " -Wno-undefined-bool-conversion"; // hides checking if 'this' pointer is null
}
if (CompilerVersionGreaterOrEqual(3, 6, 0))
{
Result += " -Wno-unused-local-typedef"; // clang is being overly strict here? PhysX headers trigger this.
Result += " -Wno-inconsistent-missing-override"; // these have to be suppressed for UE 4.8, should be fixed later.
}
if (CompilerVersionGreaterOrEqual(3, 9, 0))
{
Result += " -Wno-undefined-var-template"; // not really a good warning to disable
}
}
Result += " -Wno-unused-variable";
// this will hide the warnings about static functions in headers that aren't used in every single .cpp file
Result += " -Wno-unused-function";
// this hides the "enumeration value 'XXXXX' not handled in switch [-Wswitch]" warnings - we should maybe remove this at some point and add UE_LOG(, Fatal, ) to default cases
Result += " -Wno-switch";
Result += " -Wno-unknown-pragmas"; // Slate triggers this (with its optimize on/off pragmas)
// needed to suppress warnings about using offsetof on non-POD types.
Result += " -Wno-invalid-offsetof";
// we use this feature to allow static FNames.
Result += " -Wno-gnu-string-literal-operator-template";
if (CompileEnvironment.bEnableShadowVariableWarnings)
{
Result += " -Wshadow" + (CompileEnvironment.bShadowVariableWarningsAsErrors ? "" : " -Wno-error=shadow");
}
if (CompileEnvironment.bEnableUndefinedIdentifierWarnings)
{
Result += " -Wundef" + (CompileEnvironment.bUndefinedIdentifierWarningsAsErrors ? "" : " -Wno-error=undef");
}
//Result += " -DOPERATOR_NEW_INLINE=FORCENOINLINE";
// shipping builds will cause this warning with "ensure", so disable only in those case
if (CompileEnvironment.Configuration == CppConfiguration.Shipping)
{
Result += " -Wno-unused-value";
Result += " -fomit-frame-pointer";
}
// switches to help debugging
else if (CompileEnvironment.Configuration == CppConfiguration.Debug)
{
Result += " -fno-inline"; // disable inlining for better debuggability (e.g. callstacks, "skip file" in gdb)
Result += " -fno-omit-frame-pointer"; // force not omitting fp
Result += " -fstack-protector"; // detect stack smashing
//Result += " -fsanitize=address"; // detect address based errors (support properly and link to libasan)
}
// debug info
// bCreateDebugInfo is normally set for all configurations, including Shipping - this is needed to enable callstack in Shipping builds (proper resolution: UEPLAT-205, separate files with debug info)
if (CompileEnvironment.bCreateDebugInfo)
{
// libdwarf (from elftoolchain 0.6.1) doesn't support DWARF4
Result += " -gdwarf-3";
}
// optimization level
if (!CompileEnvironment.bOptimizeCode)
{
Result += " -O0";
}
else
{
Result += " -O2"; // warning: as of now (2014-09-28), clang 3.5.0 miscompiles PlatformerGame with -O3 (bitfields?)
}
if (CompileEnvironment.bIsBuildingDLL)
{
Result += " -fPIC";
// Use local-dynamic TLS model. This generates less efficient runtime code for __thread variables, but avoids problems of running into
// glibc/ld.so limit (DTV_SURPLUS) for number of dlopen()'ed DSOs with static TLS (see e.g. https://www.cygwin.com/ml/libc-help/2013-11/msg00033.html)
Result += " -ftls-model=local-dynamic";
}
if (CompileEnvironment.bEnableExceptions)
{
Result += " -fexceptions";
Result += " -DPLATFORM_EXCEPTIONS_DISABLED=0";
}
else
{
Result += " -fno-exceptions"; // no exceptions
Result += " -DPLATFORM_EXCEPTIONS_DISABLED=1";
}
//Result += " -v"; // for better error diagnosis
Result += ArchitectureSpecificDefines(CompileEnvironment.Architecture);
if (CrossCompiling())
{
if (UsingClang())
{
Result += String.Format(" -target {0}", CompileEnvironment.Architecture); // Set target triple
}
Result += String.Format(" --sysroot=\"{0}\"", BaseLinuxPath);
}
return Result;
}
/// <summary>
/// Sanitizes a definition argument if needed.
/// </summary>
/// <param name="definition">A string in the format "foo=bar".</param>
/// <returns></returns>
internal static string EscapeArgument(string definition)
{
string[] splitData = definition.Split('=');
string myKey = splitData.ElementAtOrDefault(0);
string myValue = splitData.ElementAtOrDefault(1);
if (string.IsNullOrEmpty(myKey)) { return ""; }
if (!string.IsNullOrEmpty(myValue))
{
if (!myValue.StartsWith("\"") && (myValue.Contains(" ") || myValue.Contains("$")))
{
myValue = myValue.Trim('\"'); // trim any leading or trailing quotes
myValue = "\"" + myValue + "\""; // ensure wrap string with double quotes
}
// replace double quotes to escaped double quotes if exists
myValue = myValue.Replace("\"", "\\\"");
}
return myValue == null
? string.Format("{0}", myKey)
: string.Format("{0}={1}", myKey, myValue);
}
static string GetCompileArguments_CPP()
{
string Result = "";
Result += " -x c++";
Result += " -std=c++14";
return Result;
}
static string GetCompileArguments_C()
{
string Result = "";
Result += " -x c";
return Result;
}
static string GetCompileArguments_MM()
{
string Result = "";
Result += " -x objective-c++";
Result += " -fobjc-abi-version=2";
Result += " -fobjc-legacy-dispatch";
Result += " -std=c++14";
return Result;
}
// Conditionally enable (default disabled) generation of information about every class with virtual functions for use by the C++ runtime type identification features
// (`dynamic_cast' and `typeid'). If you don't use those parts of the language, you can save some space by using -fno-rtti.
// Note that exception handling uses the same information, but it will generate it as needed.
static string GetRTTIFlag(CppCompileEnvironment CompileEnvironment)
{
string Result = "";
if (CompileEnvironment.bUseRTTI)
{
Result = " -frtti";
}
else
{
Result = " -fno-rtti";
}
return Result;
}
static string GetCompileArguments_M()
{
string Result = "";
Result += " -x objective-c";
Result += " -fobjc-abi-version=2";
Result += " -fobjc-legacy-dispatch";
Result += " -std=c++14";
return Result;
}
static string GetCompileArguments_PCH()
{
string Result = "";
Result += " -x c++-header";
Result += " -std=c++14";
return Result;
}
string GetLinkArguments(LinkEnvironment LinkEnvironment)
{
string Result = "";
// debugging symbols
// Applying to all configurations @FIXME: temporary hack for FN to enable callstack in Shipping builds (proper resolution: UEPLAT-205)
Result += " -rdynamic"; // needed for backtrace_symbols()...
if (LinkEnvironment.bIsBuildingDLL)
{
Result += " -shared";
}
else
{
// ignore unresolved symbols in shared libs
Result += string.Format(" -Wl,--unresolved-symbols=ignore-in-shared-libs");
}
// RPATH for third party libs
Result += " -Wl,-rpath=${ORIGIN}";
Result += " -Wl,-rpath-link=${ORIGIN}";
Result += " -Wl,-rpath=${ORIGIN}/../../../Engine/Binaries/Linux";
Result += " -Wl,-rpath=${ORIGIN}/.."; // for modules that are in sub-folders of the main Engine/Binary/Linux folder
// FIXME: really ugly temp solution. Modules need to be able to specify this
Result += " -Wl,-rpath=${ORIGIN}/../../../Engine/Binaries/ThirdParty/Steamworks/Steamv139/x86_64-unknown-linux-gnu";
if (LinkEnvironment.Architecture.StartsWith("x86_64"))
{
Result += " -Wl,-rpath=${ORIGIN}/../../../Engine/Binaries/ThirdParty/Qualcomm/Linux";
}
else
{
// x86_64 is now using updated ICU that doesn't need extra .so
Result += " -Wl,-rpath=${ORIGIN}/../../../Engine/Binaries/ThirdParty/ICU/icu4c-53_1/Linux/" + LinkEnvironment.Architecture;
}
Result += " -Wl,-rpath=${ORIGIN}/../../../Engine/Binaries/ThirdParty/OpenAL/Linux/" + LinkEnvironment.Architecture;
Result += " -Wl,-rpath=${ORIGIN}/../../../Engine/Binaries/ThirdParty/CEF3/Linux";
// Some OS ship ld with new ELF dynamic tags, which use DT_RUNPATH vs DT_RPATH. Since DT_RUNPATH do not propagate to dlopen()ed DSOs,
// this breaks the editor on such systems. See https://kenai.com/projects/maxine/lists/users/archive/2011-01/message/12 for details
Result += " -Wl,--disable-new-dtags";
// This severely improves dynamic linker performance, but affects iteration times. Do not do it in Debug,
// because taking a hit of several tens of seconds on startup is better than linking for ~4 more minutes when iterating
if (LinkEnvironment.Configuration != CppConfiguration.Debug)
{
Result += " -Wl,--as-needed";
}
// Additionally speeds up editor startup by 1-2s
Result += " -Wl,--hash-style=gnu";
if (CrossCompiling())
{
if (UsingClang())
{
Result += String.Format(" -target {0}", LinkEnvironment.Architecture); // Set target triple
}
string SysRootPath = BaseLinuxPath.TrimEnd(new char[] { '\\', '/' });
Result += String.Format(" \"--sysroot={0}\"", SysRootPath);
}
return Result;
}
static string GetArchiveArguments(LinkEnvironment LinkEnvironment)
{
return " rc";
}
public static void CrossCompileOutputReceivedDataEventHandler(Object Sender, DataReceivedEventArgs e)
{
Debug.Assert(CrossCompiling());
string Output = e.Data;
if (String.IsNullOrEmpty(Output))
{
return;
}
// format the string so the output errors are clickable in Visual Studio
// Need to match following for clickable links
string RegexFilePath = @"^[A-Z]\:([\\\/][A-Za-z0-9_\-\.]*)+\.(cpp|c|mm|m|hpp|h)";
string RegexLineNumber = @"\:\d+\:\d+\:";
string RegexDescription = @"(\serror:\s|\swarning:\s|\snote:\s).*";
// Get Matches
string MatchFilePath = Regex.Match(Output, RegexFilePath).Value.Replace("Engine\\Source\\..\\..\\", "");
string MatchLineNumber = Regex.Match(Output, RegexLineNumber).Value;
string MatchDescription = Regex.Match(Output, RegexDescription).Value;
// If any of the above matches failed, do nothing
if (MatchFilePath.Length == 0 ||
MatchLineNumber.Length == 0 ||
MatchDescription.Length == 0)
{
Console.WriteLine(Output);
return;
}
// Convert Path
string RegexStrippedPath = @"\\Engine\\.*"; //@"(Engine\/|[A-Za-z0-9_\-\.]*\/).*";
string ConvertedFilePath = Regex.Match(MatchFilePath, RegexStrippedPath).Value;
ConvertedFilePath = Path.GetFullPath("..\\.." + ConvertedFilePath);
// Extract Line + Column Number
string ConvertedLineNumber = Regex.Match(MatchLineNumber, @"\d+").Value;
string ConvertedColumnNumber = Regex.Match(MatchLineNumber, @"(?<=:\d+:)\d+").Value;
// Write output
string ConvertedExpression = " " + ConvertedFilePath + "(" + ConvertedLineNumber + "," + ConvertedColumnNumber + "):" + MatchDescription;
Console.WriteLine(ConvertedExpression); // To create clickable vs link
}
// cache the location of NDK tools
static string BaseLinuxPath;
static string ClangPath;
static string GCCPath;
static string ArPath;
static string RanlibPath;
static string StripPath;
/// <summary>
/// Version string of the current compiler, whether clang or gcc or whatever
/// </summary>
static string CompilerVersionString;
/// <summary>
/// Major version of the current compiler, whether clang or gcc or whatever
/// </summary>
static int CompilerVersionMajor = -1;
/// <summary>
/// Minor version of the current compiler, whether clang or gcc or whatever
/// </summary>
static int CompilerVersionMinor = -1;
/// <summary>
/// Patch version of the current compiler, whether clang or gcc or whatever
/// </summary>
static int CompilerVersionPatch = -1;
/// <summary>
/// Track which scripts need to be deleted before appending to
/// </summary>
private bool bHasWipedFixDepsScript = false;
/// <summary>
/// Tracks that information about used C++ library is only printed once
/// </summary>
private bool bHasPrintedLibcxxInformation = false;
private static List<FileItem> BundleDependencies = new List<FileItem>();
public override CPPOutput CompileCPPFiles(CppCompileEnvironment CompileEnvironment, List<FileItem> SourceFiles, string ModuleName, ActionGraph ActionGraph)
{
string Arguments = GetCLArguments_Global(CompileEnvironment);
string PCHArguments = "";
if (!bHasPrintedLibcxxInformation)
{
// inform the user which C++ library the engine is going to be compiled against - important for compatibility with third party code that uses STL
bool bUseLibCxx = ShouldUseLibcxx(CompileEnvironment.Architecture);
Console.WriteLine("Using {0} standard C++ library.", bUseLibCxx ? "bundled libc++" : "compiler default (most likely libstdc++)");
bHasPrintedLibcxxInformation = true;
}
if (CompileEnvironment.PrecompiledHeaderAction == PrecompiledHeaderAction.Include)
{
// Add the precompiled header file's path to the include path so Clang can find it.
// This needs to be before the other include paths to ensure Clang uses it instead of the source header file.
var PrecompiledFileExtension = UEBuildPlatform.GetBuildPlatform(UnrealTargetPlatform.Linux).GetBinaryExtension(UEBuildBinaryType.PrecompiledHeader);
PCHArguments += string.Format(" -include \"{0}\"", CompileEnvironment.PrecompiledHeaderFile.AbsolutePath.Replace(PrecompiledFileExtension, ""));
}
foreach(FileReference ForceIncludeFile in CompileEnvironment.ForceIncludeFiles)
{
PCHArguments += String.Format(" -include \"{0}\"", ForceIncludeFile.FullName);
}
// Add include paths to the argument list.
foreach (string IncludePath in CompileEnvironment.IncludePaths.UserIncludePaths)
{
Arguments += string.Format(" -I\"{0}\"", IncludePath);
}
foreach (string IncludePath in CompileEnvironment.IncludePaths.SystemIncludePaths)
{
Arguments += string.Format(" -I\"{0}\"", IncludePath);
}
// Add preprocessor definitions to the argument list.
foreach (string Definition in CompileEnvironment.Definitions)
{
Arguments += string.Format(" -D \"{0}\"", EscapeArgument(Definition));
}
// Create a compile action for each source file.
CPPOutput Result = new CPPOutput();
foreach (FileItem SourceFile in SourceFiles)
{
Action CompileAction = ActionGraph.Add(ActionType.Compile);
string FileArguments = "";
string Extension = Path.GetExtension(SourceFile.AbsolutePath).ToUpperInvariant();
// Add C or C++ specific compiler arguments.
if (CompileEnvironment.PrecompiledHeaderAction == PrecompiledHeaderAction.Create)
{
FileArguments += GetCompileArguments_PCH();
}
else if (Extension == ".C")
{
// Compile the file as C code.
FileArguments += GetCompileArguments_C();
}
else if (Extension == ".CC")
{
// Compile the file as C++ code.
FileArguments += GetCompileArguments_CPP();
}
else if (Extension == ".MM")
{
// Compile the file as Objective-C++ code.
FileArguments += GetCompileArguments_MM();
FileArguments += GetRTTIFlag(CompileEnvironment);
}
else if (Extension == ".M")
{
// Compile the file as Objective-C code.
FileArguments += GetCompileArguments_M();
}
else
{
FileArguments += GetCompileArguments_CPP();
// only use PCH for .cpp files
FileArguments += PCHArguments;
}
// Add the C++ source file and its included files to the prerequisite item list.
AddPrerequisiteSourceFile(CompileEnvironment, SourceFile, CompileAction.PrerequisiteItems);
if (CompileEnvironment.PrecompiledHeaderAction == PrecompiledHeaderAction.Create)
{
var PrecompiledFileExtension = UEBuildPlatform.GetBuildPlatform(UnrealTargetPlatform.Linux).GetBinaryExtension(UEBuildBinaryType.PrecompiledHeader);
// Add the precompiled header file to the produced item list.
FileItem PrecompiledHeaderFile = FileItem.GetItemByFileReference(
FileReference.Combine(
CompileEnvironment.OutputDirectory,
Path.GetFileName(SourceFile.AbsolutePath) + PrecompiledFileExtension
)
);
CompileAction.ProducedItems.Add(PrecompiledHeaderFile);
Result.PrecompiledHeaderFile = PrecompiledHeaderFile;
// Add the parameters needed to compile the precompiled header file to the command-line.
FileArguments += string.Format(" -o \"{0}\"", PrecompiledHeaderFile.AbsolutePath);
}
else
{
if (CompileEnvironment.PrecompiledHeaderAction == PrecompiledHeaderAction.Include)
{
CompileAction.bIsUsingPCH = true;
CompileAction.PrerequisiteItems.Add(CompileEnvironment.PrecompiledHeaderFile);
}
var ObjectFileExtension = UEBuildPlatform.GetBuildPlatform(UnrealTargetPlatform.Linux).GetBinaryExtension(UEBuildBinaryType.Object);
// Add the object file to the produced item list.
FileItem ObjectFile = FileItem.GetItemByFileReference(
FileReference.Combine(
CompileEnvironment.OutputDirectory,
Path.GetFileName(SourceFile.AbsolutePath) + ObjectFileExtension
)
);
CompileAction.ProducedItems.Add(ObjectFile);
Result.ObjectFiles.Add(ObjectFile);
FileArguments += string.Format(" -o \"{0}\"", ObjectFile.AbsolutePath);
}
// Add the source file path to the command-line.
FileArguments += string.Format(" \"{0}\"", SourceFile.AbsolutePath);
CompileAction.WorkingDirectory = UnrealBuildTool.EngineSourceDirectory.FullName;
if (!UsingClang())
{
CompileAction.CommandPath = GCCPath;
}
else
{
CompileAction.CommandPath = ClangPath;
}
CompileAction.CommandArguments = Arguments + FileArguments + CompileEnvironment.AdditionalArguments;
CompileAction.CommandDescription = "Compile";
CompileAction.StatusDescription = Path.GetFileName(SourceFile.AbsolutePath);
CompileAction.bIsGCCCompiler = true;
// Don't farm out creation of pre-compiled headers as it is the critical path task.
CompileAction.bCanExecuteRemotely =
CompileEnvironment.PrecompiledHeaderAction != PrecompiledHeaderAction.Create ||
CompileEnvironment.bAllowRemotelyCompiledPCHs;
// piping output through the handler during native builds is unnecessary and reportedly causes problems with tools like octobuild.
if (CrossCompiling())
{
CompileAction.OutputEventHandler = new DataReceivedEventHandler(CrossCompileOutputReceivedDataEventHandler);
}
}
return Result;
}
/// <summary>
/// Creates an action to archive all the .o files into single .a file
/// </summary>
public FileItem CreateArchiveAndIndex(LinkEnvironment LinkEnvironment, ActionGraph ActionGraph)
{
// Create an archive action
Action ArchiveAction = ActionGraph.Add(ActionType.Link);
ArchiveAction.WorkingDirectory = UnrealBuildTool.EngineSourceDirectory.FullName;
bool bUsingSh = BuildHostPlatform.Current.Platform != UnrealTargetPlatform.Win64 && BuildHostPlatform.Current.Platform != UnrealTargetPlatform.Win32;
if (bUsingSh)
{
ArchiveAction.CommandPath = "/bin/sh";
ArchiveAction.CommandArguments = "-c '";
}
else
{
ArchiveAction.CommandPath = "cmd.exe";
ArchiveAction.CommandArguments = "/c \"";
}
// this will produce a final library
ArchiveAction.bProducesImportLibrary = true;
// Add the output file as a production of the link action.
FileItem OutputFile = FileItem.GetItemByFileReference(LinkEnvironment.OutputFilePath);
ArchiveAction.ProducedItems.Add(OutputFile);
ArchiveAction.CommandDescription = "Archive";
ArchiveAction.StatusDescription = Path.GetFileName(OutputFile.AbsolutePath);
ArchiveAction.CommandArguments += string.Format("\"{0}\" {1} \"{2}\"", GetArPath(LinkEnvironment.Architecture), GetArchiveArguments(LinkEnvironment), OutputFile.AbsolutePath);
// Add the input files to a response file, and pass the response file on the command-line.
List<string> InputFileNames = new List<string>();
foreach (FileItem InputFile in LinkEnvironment.InputFiles)
{
string InputAbsolutePath = InputFile.AbsolutePath.Replace("\\", "/");
InputFileNames.Add(string.Format("\"{0}\"", InputAbsolutePath));
ArchiveAction.PrerequisiteItems.Add(InputFile);
}
// this won't stomp linker's response (which is not used when compiling static libraries)
FileReference ResponsePath = GetResponseFileName(LinkEnvironment, OutputFile);
if (!ProjectFileGenerator.bGenerateProjectFiles)
{
ResponseFile.Create(ResponsePath, InputFileNames);
}
ArchiveAction.CommandArguments += string.Format(" @\"{0}\"", ConvertPath(ResponsePath.FullName));
// add ranlib
ArchiveAction.CommandArguments += string.Format(" && \"{0}\" \"{1}\"", GetRanlibPath(LinkEnvironment.Architecture), OutputFile.AbsolutePath);
// Add the additional arguments specified by the environment.
ArchiveAction.CommandArguments += LinkEnvironment.AdditionalArguments;
ArchiveAction.CommandArguments = ArchiveAction.CommandArguments.Replace("\\", "/");
if (bUsingSh)
{
ArchiveAction.CommandArguments += "'";
}
else
{
ArchiveAction.CommandArguments += "\"";
}
// Only execute linking on the local PC.
ArchiveAction.bCanExecuteRemotely = false;
// piping output through the handler during native builds is unnecessary and reportedly causes problems with tools like octobuild.
if (CrossCompiling())
{
ArchiveAction.OutputEventHandler = new DataReceivedEventHandler(CrossCompileOutputReceivedDataEventHandler);
}
return OutputFile;
}
public FileItem FixDependencies(LinkEnvironment LinkEnvironment, FileItem Executable, ActionGraph ActionGraph)
{
if (!LinkEnvironment.bIsCrossReferenced)
{
return null;
}
Log.TraceVerbose("Adding postlink step");
bool bUseCmdExe = BuildHostPlatform.Current.Platform == UnrealTargetPlatform.Win64 || BuildHostPlatform.Current.Platform == UnrealTargetPlatform.Win32;
string ShellBinary = bUseCmdExe ? "cmd.exe" : "/bin/sh";
string ExecuteSwitch = bUseCmdExe ? " /C" : ""; // avoid -c so scripts don't need +x
string ScriptName = bUseCmdExe ? "FixDependencies.bat" : "FixDependencies.sh";
FileItem FixDepsScript = FileItem.GetItemByFileReference(FileReference.Combine(LinkEnvironment.LocalShadowDirectory, ScriptName));
Action PostLinkAction = ActionGraph.Add(ActionType.Link);
PostLinkAction.WorkingDirectory = UnrealBuildTool.EngineSourceDirectory.FullName;
PostLinkAction.CommandPath = ShellBinary;
PostLinkAction.StatusDescription = string.Format("{0}", Path.GetFileName(Executable.AbsolutePath));
PostLinkAction.CommandDescription = "FixDeps";
PostLinkAction.bCanExecuteRemotely = false;
PostLinkAction.CommandArguments = ExecuteSwitch;
PostLinkAction.CommandArguments += bUseCmdExe ? " \"" : " -c '";
FileItem OutputFile = FileItem.GetItemByFileReference(FileReference.Combine(LinkEnvironment.LocalShadowDirectory, Path.GetFileNameWithoutExtension(Executable.AbsolutePath) + ".link"));
// Make sure we don't run this script until the all executables and shared libraries
// have been built.
PostLinkAction.PrerequisiteItems.Add(Executable);
foreach (FileItem Dependency in BundleDependencies)
{
PostLinkAction.PrerequisiteItems.Add(Dependency);
}
PostLinkAction.CommandArguments += ShellBinary + ExecuteSwitch + " \"" + FixDepsScript.AbsolutePath + "\" && ";
// output file should not be empty or it will be rebuilt next time
string Touch = bUseCmdExe ? "echo \"Dummy\" >> \"{0}\" && copy /b \"{0}\" +,," : "echo \"Dummy\" >> \"{0}\"";
PostLinkAction.CommandArguments += String.Format(Touch, OutputFile.AbsolutePath);
PostLinkAction.CommandArguments += bUseCmdExe ? "\"" : "'";
System.Console.WriteLine("{0} {1}", PostLinkAction.CommandPath, PostLinkAction.CommandArguments);
// piping output through the handler during native builds is unnecessary and reportedly causes problems with tools like octobuild.
if (CrossCompiling())
{
PostLinkAction.OutputEventHandler = new DataReceivedEventHandler(CrossCompileOutputReceivedDataEventHandler);
}
PostLinkAction.ProducedItems.Add(OutputFile);
return OutputFile;
}
public override FileItem LinkFiles(LinkEnvironment LinkEnvironment, bool bBuildImportLibraryOnly, ActionGraph ActionGraph)
{
Debug.Assert(!bBuildImportLibraryOnly);
List<string> RPaths = new List<string>();
if (LinkEnvironment.bIsBuildingLibrary || bBuildImportLibraryOnly)
{
return CreateArchiveAndIndex(LinkEnvironment, ActionGraph);
}
// Create an action that invokes the linker.
Action LinkAction = ActionGraph.Add(ActionType.Link);
LinkAction.WorkingDirectory = UnrealBuildTool.EngineSourceDirectory.FullName;
if (String.IsNullOrEmpty(ClangPath))
{
LinkAction.CommandPath = GCCPath;
}
else
{
LinkAction.CommandPath = ClangPath;
}
// Get link arguments.
LinkAction.CommandArguments = GetLinkArguments(LinkEnvironment);
// Tell the action that we're building an import library here and it should conditionally be
// ignored as a prerequisite for other actions
LinkAction.bProducesImportLibrary = LinkEnvironment.bIsBuildingDLL;
// Add the output file as a production of the link action.
FileItem OutputFile = FileItem.GetItemByFileReference(LinkEnvironment.OutputFilePath);
LinkAction.ProducedItems.Add(OutputFile);
LinkAction.CommandDescription = "Link";
LinkAction.StatusDescription = Path.GetFileName(OutputFile.AbsolutePath);
// Add the output file to the command-line.
LinkAction.CommandArguments += string.Format(" -o \"{0}\"", OutputFile.AbsolutePath);
// Add the input files to a response file, and pass the response file on the command-line.
List<string> InputFileNames = new List<string>();
foreach (FileItem InputFile in LinkEnvironment.InputFiles)
{
InputFileNames.Add(string.Format("\"{0}\"", InputFile.AbsolutePath.Replace("\\", "/")));
LinkAction.PrerequisiteItems.Add(InputFile);
}
FileReference ResponseFileName = GetResponseFileName(LinkEnvironment, OutputFile);
LinkAction.CommandArguments += string.Format(" -Wl,@\"{0}\"", ResponseFile.Create(ResponseFileName, InputFileNames));
if (LinkEnvironment.bIsBuildingDLL)
{
LinkAction.CommandArguments += string.Format(" -Wl,-soname={0}", OutputFile);
}
// Start with the configured LibraryPaths and also add paths to any libraries that
// we depend on (libraries that we've build ourselves).
List<string> AllLibraryPaths = LinkEnvironment.LibraryPaths;
foreach (string AdditionalLibrary in LinkEnvironment.AdditionalLibraries)
{
string PathToLib = Path.GetDirectoryName(AdditionalLibrary);
if (!String.IsNullOrEmpty(PathToLib))
{
// make path absolute, because FixDependencies script may be executed in a different directory
string AbsolutePathToLib = Path.GetFullPath(PathToLib);
if (!AllLibraryPaths.Contains(AbsolutePathToLib))
{
AllLibraryPaths.Add(AbsolutePathToLib);
}
}
if ((AdditionalLibrary.Contains("Plugins") || AdditionalLibrary.Contains("Binaries/ThirdParty") || AdditionalLibrary.Contains("Binaries\\ThirdParty")) && Path.GetDirectoryName(AdditionalLibrary) != Path.GetDirectoryName(OutputFile.AbsolutePath))
{
string RelativePath = Utils.MakePathRelativeTo(Path.GetDirectoryName(AdditionalLibrary), Path.GetDirectoryName(OutputFile.AbsolutePath));
if (!RPaths.Contains(RelativePath))
{
RPaths.Add(RelativePath);
LinkAction.CommandArguments += string.Format(" -Wl,-rpath=\"${{ORIGIN}}/{0}\"", RelativePath);
}
}
}
LinkAction.CommandArguments += string.Format(" -Wl,-rpath-link=\"{0}\"", Path.GetDirectoryName(OutputFile.AbsolutePath));
// Add the library paths to the argument list.
foreach (string LibraryPath in AllLibraryPaths)
{
// use absolute paths because of FixDependencies script again
LinkAction.CommandArguments += string.Format(" -L\"{0}\"", Path.GetFullPath(LibraryPath));
}
List<string> EngineAndGameLibraries = new List<string>();
// Pre-2.25 ld has symbol resolution problems when .so are mixed with .a in a single --start-group/--end-group
// when linking with --as-needed.
// Move external libraries to a separate --start-group/--end-group to fix it (and also make groups smaller and faster to link).
// See https://github.com/EpicGames/UnrealEngine/pull/2778 and https://github.com/EpicGames/UnrealEngine/pull/2793 for discussion
string ExternalLibraries = "";
// add libraries in a library group
LinkAction.CommandArguments += string.Format(" -Wl,--start-group");
foreach (string AdditionalLibrary in LinkEnvironment.AdditionalLibraries)
{
if (String.IsNullOrEmpty(Path.GetDirectoryName(AdditionalLibrary)))
{
// library was passed just like "jemalloc", turn it into -ljemalloc
ExternalLibraries += string.Format(" -l{0}", AdditionalLibrary);
}
else if (Path.GetExtension(AdditionalLibrary) == ".a")
{
// static library passed in, pass it along but make path absolute, because FixDependencies script may be executed in a different directory
string AbsoluteAdditionalLibrary = Path.GetFullPath(AdditionalLibrary);
if (AbsoluteAdditionalLibrary.Contains(" "))
{
AbsoluteAdditionalLibrary = string.Format("\"{0}\"", AbsoluteAdditionalLibrary);
}
// libcrypto/libssl contain number of functions that are being used in different DSOs. FIXME: generalize?
if (LinkEnvironment.bIsBuildingDLL && (AbsoluteAdditionalLibrary.Contains("libcrypto") || AbsoluteAdditionalLibrary.Contains("libssl")))
{
LinkAction.CommandArguments += (" -Wl,--whole-archive " + AbsoluteAdditionalLibrary + " -Wl,--no-whole-archive");
}
else
{
LinkAction.CommandArguments += (" " + AbsoluteAdditionalLibrary);
}
LinkAction.PrerequisiteItems.Add(FileItem.GetItemByPath(AdditionalLibrary));
}
else
{
// Skip over full-pathed library dependencies when building DLLs to avoid circular
// dependencies.
FileItem LibraryDependency = FileItem.GetItemByPath(AdditionalLibrary);
var LibName = Path.GetFileNameWithoutExtension(AdditionalLibrary);
if (LibName.StartsWith("lib"))
{
// Remove lib prefix
LibName = LibName.Remove(0, 3);
}
string LibLinkFlag = string.Format(" -l{0}", LibName);
if (LinkEnvironment.bIsBuildingDLL && LinkEnvironment.bIsCrossReferenced)
{
// We are building a cross referenced DLL so we can't actually include
// dependencies at this point. Instead we add it to the list of
// libraries to be used in the FixDependencies step.
EngineAndGameLibraries.Add(LibLinkFlag);
if (!LinkAction.CommandArguments.Contains("--allow-shlib-undefined"))
{
LinkAction.CommandArguments += string.Format(" -Wl,--allow-shlib-undefined");
}
}
else
{
LinkAction.PrerequisiteItems.Add(LibraryDependency);
ExternalLibraries += LibLinkFlag;
}
}
}
LinkAction.CommandArguments += " -Wl,--end-group";
LinkAction.CommandArguments += " -Wl,--start-group";
LinkAction.CommandArguments += ExternalLibraries;
LinkAction.CommandArguments += " -Wl,--end-group";
LinkAction.CommandArguments += " -lrt"; // needed for clock_gettime()
LinkAction.CommandArguments += " -lm"; // math
if (ShouldUseLibcxx(LinkEnvironment.Architecture))
{
// libc++ and its abi lib
LinkAction.CommandArguments += " -nodefaultlibs";
LinkAction.CommandArguments += " -L" + "ThirdParty/Linux/LibCxx/lib/Linux/" + LinkEnvironment.Architecture + "/";
LinkAction.CommandArguments += " " + "ThirdParty/Linux/LibCxx/lib/Linux/" + LinkEnvironment.Architecture + "/libc++.a";
LinkAction.CommandArguments += " " + "ThirdParty/Linux/LibCxx/lib/Linux/" + LinkEnvironment.Architecture + "/libc++abi.a";
LinkAction.CommandArguments += " -lm";
LinkAction.CommandArguments += " -lc";
LinkAction.CommandArguments += " -lgcc_s";
LinkAction.CommandArguments += " -lgcc";
}
// these can be helpful for understanding the order of libraries or library search directories
//LinkAction.CommandArguments += " -Wl,--verbose";
//LinkAction.CommandArguments += " -Wl,--trace";
//LinkAction.CommandArguments += " -v";
// Add the additional arguments specified by the environment.
LinkAction.CommandArguments += LinkEnvironment.AdditionalArguments;
LinkAction.CommandArguments = LinkAction.CommandArguments.Replace("\\\\", "/");
LinkAction.CommandArguments = LinkAction.CommandArguments.Replace("\\", "/");
// prepare a linker script
FileReference LinkerScriptPath = FileReference.Combine(LinkEnvironment.LocalShadowDirectory, "remove-sym.ldscript");
if (!DirectoryReference.Exists(LinkEnvironment.LocalShadowDirectory))
{
DirectoryReference.CreateDirectory(LinkEnvironment.LocalShadowDirectory);
}
if (FileReference.Exists(LinkerScriptPath))
{
FileReference.Delete(LinkerScriptPath);
}
using (StreamWriter Writer = File.CreateText(LinkerScriptPath.FullName))
{
Writer.WriteLine("UE4 {");
Writer.WriteLine(" global: *;");
Writer.WriteLine(" local: _Znwm;");
Writer.WriteLine(" _ZnwmRKSt9nothrow_t;");
Writer.WriteLine(" _Znam;");
Writer.WriteLine(" _ZnamRKSt9nothrow_t;");
Writer.WriteLine(" _ZdaPv;");
Writer.WriteLine(" _ZdaPvRKSt9nothrow_t;");
Writer.WriteLine(" _ZdlPv;");
Writer.WriteLine(" _ZdlPvRKSt9nothrow_t;");
// Hide ALL std:: symbols as they can collide
// with any c++ library out there (like LLVM)
// and this may lead to operator new/delete mismatches
Writer.WriteLine(" _ZNSt8*;");
Writer.WriteLine("};");
};
LinkAction.CommandArguments += string.Format(" -Wl,--version-script=\"{0}\"", LinkerScriptPath);
// Only execute linking on the local PC.
LinkAction.bCanExecuteRemotely = false;
// Prepare a script that will run later, once all shared libraries and the executable
// are created. This script will be called by action created in FixDependencies()
if (LinkEnvironment.bIsCrossReferenced && LinkEnvironment.bIsBuildingDLL)
{
bool bUseCmdExe = BuildHostPlatform.Current.Platform == UnrealTargetPlatform.Win64 || BuildHostPlatform.Current.Platform == UnrealTargetPlatform.Win32;
string ScriptName = bUseCmdExe ? "FixDependencies.bat" : "FixDependencies.sh";
string FixDepsScriptPath = Path.Combine(LinkEnvironment.LocalShadowDirectory.FullName, ScriptName);
if (!bHasWipedFixDepsScript)
{
bHasWipedFixDepsScript = true;
Log.TraceVerbose("Creating script: {0}", FixDepsScriptPath);
StreamWriter Writer = File.CreateText(FixDepsScriptPath);
if (bUseCmdExe)
{
Writer.Write("@echo off\n");
Writer.Write("rem Automatically generated by UnrealBuildTool\n");
Writer.Write("rem *DO NOT EDIT*\n\n");
}
else
{
Writer.Write("#!/bin/sh\n");
Writer.Write("# Automatically generated by UnrealBuildTool\n");
Writer.Write("# *DO NOT EDIT*\n\n");
Writer.Write("set -o errexit\n");
}
Writer.Close();
}
StreamWriter FixDepsScript = File.AppendText(FixDepsScriptPath);
string EngineAndGameLibrariesString = "";
foreach (string Library in EngineAndGameLibraries)
{
EngineAndGameLibrariesString += Library;
}
FixDepsScript.Write(string.Format("echo Fixing {0}\n", Path.GetFileName(OutputFile.AbsolutePath)));
if (!bUseCmdExe)
{
FixDepsScript.Write(string.Format("TIMESTAMP=`stat --format %y \"{0}\"`\n", OutputFile.AbsolutePath));
}
string FixDepsLine = LinkAction.CommandPath + " " + LinkAction.CommandArguments;
string Replace = "-Wl,--allow-shlib-undefined";
FixDepsLine = FixDepsLine.Replace(Replace, EngineAndGameLibrariesString);
string OutputFileForwardSlashes = OutputFile.AbsolutePath.Replace("\\", "/");
FixDepsLine = FixDepsLine.Replace(OutputFileForwardSlashes, OutputFileForwardSlashes + ".fixed");
FixDepsLine = FixDepsLine.Replace("$", "\\$");
FixDepsScript.Write(FixDepsLine + "\n");
if (bUseCmdExe)
{
FixDepsScript.Write(string.Format("move /Y \"{0}.fixed\" \"{0}\"\n", OutputFile.AbsolutePath));
}
else
{
FixDepsScript.Write(string.Format("mv \"{0}.fixed\" \"{0}\"\n", OutputFile.AbsolutePath));
FixDepsScript.Write(string.Format("touch -d \"$TIMESTAMP\" \"{0}\"\n\n", OutputFile.AbsolutePath));
}
FixDepsScript.Close();
}
if (CrossCompiling())
{
LinkAction.OutputEventHandler = new DataReceivedEventHandler(CrossCompileOutputReceivedDataEventHandler);
}
return OutputFile;
}
public override void CompileCSharpProject(CSharpEnvironment CompileEnvironment, FileReference ProjectFileName, FileReference DestinationFile, ActionGraph ActionGraph)
{
throw new BuildException("Linux cannot compile C# files");
}
public override void SetupBundleDependencies(List<UEBuildBinary> Binaries, string GameName)
{
foreach (UEBuildBinary Binary in Binaries)
{
BundleDependencies.Add(FileItem.GetItemByFileReference(Binary.Config.OutputFilePath));
}
}
/// <summary>
/// Converts the passed in path from UBT host to compiler native format.
/// </summary>
public static string ConvertPath(string OriginalPath)
{
if (BuildHostPlatform.Current.Platform == UnrealTargetPlatform.Linux)
{
return OriginalPath.Replace("\\", "/");
}
else
{
return OriginalPath;
}
}
public override ICollection<FileItem> PostBuild(FileItem Executable, LinkEnvironment BinaryLinkEnvironment, ActionGraph ActionGraph)
{
var OutputFiles = base.PostBuild(Executable, BinaryLinkEnvironment, ActionGraph);
if (BinaryLinkEnvironment.bIsBuildingDLL || BinaryLinkEnvironment.bIsBuildingLibrary)
{
return OutputFiles;
}
FileItem FixDepsOutputFile = FixDependencies(BinaryLinkEnvironment, Executable, ActionGraph);
if (FixDepsOutputFile != null)
{
OutputFiles.Add(FixDepsOutputFile);
}
return OutputFiles;
}
public void StripSymbols(FileReference SourceFile, FileReference TargetFile)
{
if (SourceFile != TargetFile)
{
// Strip command only works in place so we need to copy original if target is different
File.Copy(SourceFile.FullName, TargetFile.FullName, true);
}
ProcessStartInfo StartInfo = new ProcessStartInfo();
StartInfo.FileName = GetStripPath(Architecture);
StartInfo.Arguments = "--strip-debug \"" + TargetFile.FullName + "\"";
StartInfo.UseShellExecute = false;
StartInfo.CreateNoWindow = true;
Utils.RunLocalProcessAndLogOutput(StartInfo);
}
}
}