Commit Graph

38 Commits

Author SHA1 Message Date
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
jonathan adamczewski
f270855eef AutomationTool: Compile script modules within the application
Add a layer of caching to avoid running msbuild as much as possible.

#jira UE-109181
#rb ben.marsh

#ROBOMERGE-SOURCE: CL 17102399 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17102408 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-09 10:39:35 -04:00
david hibbitts
c626c82f43 Updated automation mappings for XLOC
[FYI] Jamie.Dale

#ROBOMERGE-SOURCE: CL 16914497 via CL 16920907
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16920991 by david hibbitts in ue5-release-engine-test branch]
2021-07-22 01:03:00 -04:00
nick darnell
9b0d5728ce Localization - Adding CrowdinLocalization provider, basic support implemented. Updated the API for localization providers to make async/await easier to implement.
#jira none
Jamie.Dale

#ROBOMERGE-SOURCE: CL 16912307 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16912319 by nick darnell in ue5-release-engine-test branch]
2021-07-21 14:16:41 -04:00
Ben Marsh
6f927647b1 Changing C# code over to using EpicGames.Core over DotNETCommon.
#rb none
#rnx

[CL 14962096 by Ben Marsh in ue5-main branch]
2020-12-21 23:07:37 -04:00
Joakim Lindqvist
017048f505 Changed pdb format to not use portable pdbs (which is the default in net core) and instead back to our old pdb configuration.
This resolves a issue with PCBs were pdbcopy failed to run because it does not support portable pdbs.

#rb none

[CL 14837521 by Joakim Lindqvist in ue5-main branch]
2020-12-02 14:54:53 -04:00
Joakim Lindqvist
e7039d3d35 UBT and UAT now use .NET Core instead of Framework and Mono. This means that we use the same runtime on Windows, Linux and Mac. Further benefits including newer C# features and a lot of intresting features for the future around AOT and Tiered compilation.
Some behavior changes:
Output paths - Both tools are now output to a subdirectory of Binaries/Dotnet, I believe most hardcoded paths have been fixed up but there may be tools that will fail because of this.
UAT Plugin Building - As .NET Core does not support AppDomain unloading, how we build the plugins has changed quite a bit, these are now built before UAT is started rather then by UAT itself. If you just start UAT via RunUAT.bat/sh this should just continue to work.

#rb ben.marsh

[CL 14834347 by Joakim Lindqvist in ue5-main branch]
2020-12-02 06:57:13 -04:00
Joakim Lindqvist
1b4f6b3ac9 Changed output directory for UAT projects under netcore, now outputting to a seperate directory for each addin. This avoids the risk of output stomping each other breaking incremental builds. Furthermore it makes it easier to understand who adds what dependency to the output.
Also cleaned up some output assemblies, preventing all transative references from being outputed (as we assume they build directly to the output directory instead). Unfortunatley the UBT references still cause tranastive dependencies to be copied as this is also a executable that needs to work outside of UAT as well, we should eventually move all UBT referenced code into BuildUtilities.

#rb none
#fyi ben.marsh

[CL 14775171 by Joakim Lindqvist in ue5-main branch]
2020-11-18 10:18:50 -04:00
Joakim Lindqvist
55025e031c Moved UAT plugin building to a seperate step controlled by AutomationTool.proj, this makes the process of building addins identical for installed builds and engine builds (but we still apply seperate logic to them).
Added CopyLocalLockFileAssemblies to some addins that added new dependencies (nuget packages) to make sure these gets added to the output folder.

UAT now have a dependency on WindowsForms (on Windows) to enable the WinForms usage in Gauntlet and Turnkey, but is still a console application so just making sure the right assemblies are referenced.

Fixed up shootergame csproj were I had accidentally removed EngineDir props which would have prevented it from working for installed builds as a foreign project.

#rb ben.marsh

[CL 14773885 by Joakim Lindqvist in ue5-main branch]
2020-11-18 08:46:54 -04:00
Joakim Lindqvist
bd18d9e6e8 Disabled copy local on project references for netcore projects, fixes build warnigns caused by multiple copies trying to write to the same output files.
Also reduces footprint on disk and speedsup builds slightly.

Also took a pass over all projects to fixup their output paths as there were some inconsitencies from my port to netcore.
Lastly I disabled the resource folders for everything but english to reduce some of the noise in the output folder, this just impacts which languages you get code analysis.

#jira UE-102147
#rb none

[CL 14651855 by Joakim Lindqvist in ue5-main branch]
2020-11-04 09:48:49 -04:00
Joakim Lindqvist
f90e40ffb0 UAT can now build as a netcore application.
Added a NET_CORE define to allow us to have changes side by side.
The AWS S3 changes are required due to us requiring to upgrade the S3 assembly version to get net core support (which made all methods async).
The ACL checks for files are not available in the system libraries of net core, as such the api is a bit different.

AutomationToolLauncher now just spawns a subprocess when used in netcore, as netcore does not support custom AppDomains and shadow copying. We will generally need to revisit this for netcore as this whole feature of building the source for UAT in UAT is not really possible.

To enable this set environment variable "UE_USE_DOTNET=1", note that with netcore all applications change their output path so this will likely break a bit of tooling when enabled.

#rb ben.marsh

[CL 14572339 by Joakim Lindqvist in ue5-main branch]
2020-10-26 06:08:59 -04:00
Joakim Lindqvist
2750553a40 UAT Net Core - Copied project files into net core compatible format
#rb none

[CL 14572287 by Joakim Lindqvist in ue5-main branch]
2020-10-26 06:05:41 -04:00
Ryan Durand
9ef3748747 Updating copyrights for Engine Programs.
#rnx
#rb none
#jira none

#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869242 in //Fortnite/Release-12.00/... via CL 10869536
#ROBOMERGE-BOT: FORTNITE (Main -> Dev-EngineMerge) (v613-10869866)

[CL 10870955 by Ryan Durand in Main branch]
2019-12-26 23:01:54 -05:00
Ben Marsh
0cc6e3dca6 Copying //UE4/Dev-Build to Dev-Main (//UE4/Dev-Main)
#rb none
#rnx

[CL 6631504 by Ben Marsh in Main branch]
2019-05-24 11:51:54 -04:00
Jamie Dale
a27aefcf57 Changed some of the localization automation to use classes with read-only members, rather than use structs
#jira UE-65600
#rb Francis.Hurteau
#lockdown Matt.Kuhlenschmidt

[CL 4762550 by Jamie Dale in Dev-Editor branch]
2019-01-21 12:57:05 -05:00
Jamie Dale
adc2fefb19 Added support for splitting off per-platform localization
This is handled by the ELocTextPlatformSplitMode enum, used by FLocTextHelper (default is ELocTextPlatformSplitMode::None to preserve old behavior), and allows you to optionally split localization data for all platforms, or for any NDA restricted platforms. Note: Any localization data that is used by multiple platforms is still considered platform agnostic, even if it appears only within platform specific files.

#jira UE-65600
#rb Francis.Hurteau
#lockdown Matt.Kuhlenschmidt

[CL 4761508 by Jamie Dale in Dev-Editor branch]
2019-01-21 11:07:47 -05:00
Chris Gagnon
1dd3e0189f Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4730305 by Chris Gagnon in Dev-Editor branch]
2019-01-15 18:47:22 -05:00
Ben Marsh
0f885bbf64 Switch UAT and UBT to use .NET 4.6.2, in order to support long file names (which is enabled via an app.manifest file in each application).
#rb none

[CL 4679962 by Ben Marsh in Dev-Build branch]
2019-01-03 16:47:48 -05:00
Chris Gagnon
8fc25ea18e Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4676797 by Chris Gagnon in Dev-Editor branch]
2019-01-02 14:54:39 -05:00
Ben Marsh
530369c613 Merging //UE4/Dev-Main to Dev-Build (//UE4/Dev-Build)
#rb none
#rnx

[CL 4662695 by Ben Marsh in Dev-Build branch]
2018-12-14 14:49:12 -05:00
Ben Marsh
b0da1f4fe7 Merging //UE4/Dev-Main to Dev-Build (//UE4/Dev-Build)
#rb none

[CL 4577636 by Ben Marsh in Dev-Build branch]
2018-11-19 10:12:17 -05:00
Matt Kuhlenschmidt
cde8160913 Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4572510 by Matt Kuhlenschmidt in Dev-Editor branch]
2018-11-16 11:15:08 -05:00
Marc Audy
af90b7bcd4 Copying //UE4/Fortnite-Staging to Dev-Main (//UE4/Dev-Main) @ 4395008
#rb
#rnx
#lockdown Nick.Penwarden

[CL 4395058 by Marc Audy in Main branch]
2018-09-25 10:11:35 -04:00
Jamie Dale
33fdac4526 Added logging if XLOC fails to find a PO file to use, and now compare the PO file names case-insensitively
#jira UE-63776
#rb Arthur.Flew
#lockdown Cristina.Riveron

[CL 4348715 by Jamie Dale in Main branch]
2018-09-06 15:15:26 -04:00
Leigh Swift
159321a308 Copying //UE4/Dev-Main to //UE4/Main (Source: //Portal/Main/Engine @ 4247640)
#rb none
#lockdown Nick.Penwarden

============================
  MAJOR FEATURES & CHANGES
============================

Change 4247640 by Daniel.Lamb

	BuildPatchTool: PackageChunks: Fixed issue with filenames not being set for chunkdbs in particular cases where the chunkdbs were small.

Change 4247246 by Leigh.Swift

	BuildPatchTool: Adding support to BPT PackageChunks to filter by tagsets, and also split chunkdb output by them too.

Change 4173518 by Wes.Fudala

	WebBrowser: Adding support to enable allowing net security expirations.

Change 4102451 by Leigh.Swift

	BuildPatchTool: PackageChunks: Adding tool mode JSON output for listing created chunkdb files.

Change 4099951 by Leigh.Swift

	BuildPatchTool: PackageChunks: Adding PrevManifestFile commandline support to BPT PackageChunks mode, allowing us to generate chunkdbs that only cover data required to perform an update.

Change 4051406 by Leigh.Swift

	BuildPatchServices: Update default number of simultaneous downloads for an installer to 16 which is now well proven in the wild.

Change 4036807 by Leigh.Swift

	BuildPatchServices: Added new message type for receiving updates about which files have been touched by the installation.
	BuildPatchServices: Cloud enumeration is now threaded to improve speed.

Change 4036477 by Richard.Fawcett

	Thirdparty: AWSSDK: Update AWS SDK to version 3.3 as speculative fix for object is used after being disposed error.

	Confirmed this exception is being raised from within the AWS SDK, and _not_ in Epic code.
	Advice by Amazon on AWS forums for people experiencing this exception is always to update to the latest version of the SDK in the first instance.

Change 3922493 by Justin.Sargent

	Runtime: Added shortcircuit support to the ExpressionParser. More documentation to come.
	AgreementExpressionEvaluator has been updated to use the shortcircuit logic so it now processes expressions lef to right as expected in all cases.
	AgreementExpressionEvaluator was also updated to perform evaluation as a two pass system. The tow pass solution prevents over prompting eulas in certain scenarios where the expression would ultimately resolve true without them.

Change 3983713 by Barnabas.McManners

	BuildPatchServices: Fix for verification logging error counts for unique runs. Solved the issue by using an external cache of the errors encountered each run to deduct from the total. Reporting only the delta.

Change 3966915 by Wes.Fudala

	WebBrowser: CEF: Potential fix for a rare issue encountered when we attempting to release resources outside of the game thread.

Change 3955168 by Justin.Sargent

	BuildPatchServices: Updated primary messaging of overall install status to now display Updating when patching an existing installation rather than Installing.

Change 3954610 by Leigh.Swift

	BuildPatchServices: Fixing issue with Cloud Chunk Source which would erroneously conclude that it needed to re-download a chunk due to external system failure.

	There is now an explicit concept of 'lost chunk' which is tracked and bubbled up by the system errors that cause the loss in the first place, so that each source knows exactly when it should be required to retrieve a chunk that it had already retrieved previously. Being explicit, these lost chunks can also now contribute to the total download required stat.
	Also fixing some tracking that was missing to update the total download required stat if a local Install Chunk Source failed to load data from the files on disk and so these chunks needed to be additionally downloaded.

Change 3947928 by Chad.Garyet

	UAT: Changing commandutils to attempt to find the Win8.1sdk signtool before the win10 one.  There's currently a bug related to vs2017/server2012r2/win10sdk signtool that causes it to exit with an undefined error when signing from a service account.  Using the win8.1sdk circumvents this issue.

Change 3942776 by Rob.Cannaday

	Http: Fix for Mac sending up duplicate header strings

Change 3940306 by Leigh.Swift

	BuildPatchServices: Refactor to isolate CoreUObject dependency and be able to compile out usage.

Change 3936655 by Justin.Sargent

	Slate: Changed the invalid fontcache ensure in ShapedTextCache from always to only once to reduce ensure spamChange 3917840 by Leigh.Swift

	BuildPatchTool: DiffManifests mode now also saves info for New, Removed, Changed, and Unchanged file to the output json file.

Change 3911756 by Justin.Sargent

	WebBrowser: SWebBrowserView now ensures that it has a valid pointer to it's parent window in it's onpaint to avoid issues with the first frame being scaled incorrectly due to not being able to access the parent windows dpi scaling.

Change 3906670 by Justin.Sargent

	Slate: Change the ShapedTextCache to hold a weakptr to the Slate FontCache instead of a reference. This will allow it to detect if the FontCache has become invalid since it was linked with the ShapedTextCache, thus making it able to avoid crashing.

Change 3889008 by Justin.Sargent

	StandaloneRenderer: Made the SlateD3DRenderingPolicy more resilient to graphics device errors.

Change 3886969 by Justin.Sargent

	StandaloneRenderer:Changed SlateD3DConstantBuffer to no longer check on a D3DDevice failure and instead soft fail, so the application can go through the process of attempting to re-establish the D3DDevice.

Change 3886960 by Justin.Sargent

	WebBrowser: Made CEFWebBrowserWindow more resilient to issues with creating textures.

Change 3855821 by Barnabas.McManners

	BuildPatchServices: Added logging of the configuration to the start of all installs.

Change 3839245 by Wes.Fudala

	WebBrowser: Adding support for web browser drag regions.  These are areas of a page tagged with -webkit-app-region: drag or -webkit-app-region: no-drag.  The application can now pass a handler function to the browser to handle window drag events.  This handler will be called if the browser detects mouse drag events inside of a tagged drag region.

Change 3835225 by Jacob.Hedges

	Slate: Fix for SScrollBarTrack size issue

Change 3824320 by Wes.Fudala

	WebBrowser: Fix for reported deadlock in WebBrowserSingleton.  Associated with github pull request #4303.
	#jira UE-53420 GitHub 4330 : Fixed deadlock in FWebBrowserSingleton
	#4303

Change 3811191 by Barnabas.McManners

	BuildPatchServices: Expanded MF01-X into MF01-X-X and MF02-X where X is the os error codes

Change 3807662 by Barnabas.McManners

	BuildPatchServices: Broke down build verification errors into 4 new cases.

Change 3805698 by Leigh.Swift

	BuildPatchServices: Speculative fixes for Unit test crashes / failures.

Change 3804175 by Wes.Fudala

	ThirdParty: CEF: Adding browser locale pak files for es-MX, and es-ES as the typical mapping/fallback does not seem functional on mac browser.  They are copies of es_419 and es respectively.

Change 3786628 by Leigh.Swift

	WebBrowser: Exposing ability to customise tool tip widgets produced from SWebBrowserView.

Change 3775678 by Richard.Fawcett

	BuildPatchServices: Allow a Prerequisite install only mode.

Change 3774365 by Justin.Sargent

	BuildPatchServices: Updated Build Stat report that is logged after every installation to use FText::AsMemory instead of the UnitConversion logic, and now it outputs multiple unit types for convenience.

Change 3774361 by Justin.Sargent

	Http: Remove Pragma: no-cache header from libcurl requests

Change 3774258 by Leigh.Swift

	BuildPatchServices: Fix for destructive patch destroy files that contain useful data.

Change 3766156 by Barnabas.McManners

	Http: Various lower changes to enable Hardware testing and to enable proxy configuration.

Change 3756723 by Leigh.Swift

	BuildPatchServices: Hooking up disk chunk store operation states to the installer statistics.
	BuildPatchServices: Memory chunk store statistics fix for booted chunks that have been reloaded.

Change 3756320 by Rob.Cannaday

	Http: Add default headers added to every HTTP request.

Change 3741274 by Wes.Fudala

	WebBrowser: Release CEF related references prior to CEF shutdown.

Change 3738003 by Leigh.Swift

	BuildPatchServices: Fix-ups for install stats when failures are occurring
	NumFilesOutdated now only set on first run, so it is not set to the number of files that are retried.
	Total downloaded data and total download requirement stats fixed up for runtime as well as final values.
	Initial chunk counters now only set for first run.
	Moving the GetBytesDownloaded api from cloud source to download service so that it correctly accumulates.
	InstallSource was multiply attempting, and counting, recycle failures, throwing that stat out of proportion.

Change 3729851 by Barnabas.McManners

	BuildPatchServices: Changed the installer's MoveFile method to default to not retry. We currently only have uses of move file which already handle retry.

Change 3725611 by Leigh.Swift

	Core: FText::AsMemory - Fix for numerical edges.
	Added unit tests to check all edges up to full uint64 range.

Change 3725127 by Leigh.Swift

	BuildPatchServices: ManifestDiff: Correcting string format padding for new data size output uints.

Change 3725126 by Leigh.Swift

	Core: FText::AsMemory fix. Shifting (equivalent of divide 2 per shift) does not work for calculating SI units which are base 10.

Change 3721926 by Justin.Sargent

	ThirdParty: LibCurl update performed by Simon Tourangeau.
	We now have 100MB/sec download speed with libcurl on Win64, compared to 3MB/sec originally

Change 3700670 by Michael.Trepka

	SlateReflector: Fixed mouse click highlighting in Widget Reflector's Demo Mode in high DPI

Change 3697526 by Leigh.Swift

	BuildPatchServices: Exposing a suite of runtime statistics for BuildPatchInstallers.

Change 3686439 by Leigh.Swift

	BuildPatchServices: Stop installers from always logging a shutdown error on destruction. This should only occur if the installer is actually running.

Change 3684747 by Leigh.Swift

	BuildPatchTool: Fix file ignore list to parse using platform agnostic method.

Change 3643038 by Michael.Trepka

	Core: Don't defer Cocoa calls in FMacWindow Show and Hide to make sure both actions complete before we exit these functions. This solves the problem with the blocks being called after window was destroyed.

Change 3639692 by Michael.Trepka

	Fixes for a couple of issues found by address sanitizer

Change 3625568 by Leigh.Swift

	BuildPatchServices: Fixing numerical limits problem with double -> uint64 in FStatsCollector::SecondsToCycles().

Change 3617948 by Leigh.Swift

	BuildPatchServices: Disk space requirement can now be lower when patching if destructive patch mode is enabled. This mode will delete existing old files once they are not needed.
	BuildPatchServices: Adding new installation mode setting on installer config.
	BuildPatchServices: If destructive installation mode is enabled, the file constructor will delete old existing files after completing the new one.
	BuildPatchServices: Adding missing file path length check for install location to cover situation where staging directory is outside the install directory.

Change 3593632 by Leigh.Swift

	BuildPatchServices: Adding additional installation tracking to BuildPatchServices. See Engine/Source/Runtime/Online/BuildPatchServices/Public/Interfaces/IBuildInstaller.h

[CL 4273704 by Leigh Swift in Main branch]
2018-08-09 17:55:56 -04:00