Matt.Kuhlenschmidt
#ROBOMERGE-SOURCE: CL 9421835 via CL 9421841 via CL 9421844 via CL 9421851 via CL 9422758
#ROBOMERGE-BOT: (v480-9420520)
[CL 9423104 by nick darnell in Main branch]
#ROBOMERGE-SOURCE: CL 9388509 via CL 9388536 via CL 9388543 via CL 9388574 via CL 9388597
#ROBOMERGE-BOT: (v456-9359915)
[CL 9391943 by nick darnell in Main branch]
Replicated from CL# 7924370.
#rb none
#ROBOMERGE-OWNER: steve.robb
#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 9279060 via CL 9279063
#ROBOMERGE-BOT: (v443-9013191)
[CL 9279836 by steve robb in Main branch]
Most UE4 platforms use a 2-byte TCHAR, however some still use a 4-byte TCHAR. The platforms that use a 4-byte TCHAR expect their string data to be UTF-32, however there are parts of UE4 that serialize FString data as a series of UCS2CHAR, simply narrowing or widening each TCHAR in turn. This can result in invalid or corrupted UTF-32 strings (either UTF-32 strings containing UTF-16 surrogates, or UTF-32 code points that have been truncated to 2-bytes), which leads to either odd behavior or crashes.
This change updates the parts of UE4 that process FString data as a series of 2-byte values to do so on the correct UTF-16 interpretation of the data, converting to/from UTF-32 as required on platforms that use a 4-byte TCHAR. This conversion is a no-op on platforms that use a 2-byte TCHAR as the string is already assumed to be valid UTF-16 data. It should also be noted that while FString may contain UTF-16 code units on platforms using a 2-byte TCHAR, this change doesn't do anything to make FString represent a Unicode string on those platforms (ie, a string that understands and works on code points), but is rather just a bag of code units.
Two new variable-width string converters have be added to facilitate the conversion (modelled after the TCHAR<->UTF-8 converters), TUTF16ToUTF32_Convert and TUTF32ToUTF16_Convert. These are used for both TCHAR<->UTF16CHAR conversion when needed, but also for TCHAR<->wchar_t conversion on platforms that use char16_t for TCHAR along with having a 4-byte wchar_t (as defined by the new PLATFORM_WCHAR_IS_4_BYTES option).
These conversion routines are accessed either via the conversion macros (TCHAR_TO_UTF16, UTF16_TO_TCHAR, TCHAR_TO_WCHAR, and WCHAR_TO_TCHAR), or by using a conversion struct (FTCHARToUTF16, FUTF16ToTCHAR, FTCHARToWChar, and FWCharToTCHAR), which is the same pattern as the existing TCHAR<->UTF-8 conversion. Both the macros and the structs are defined as no-ops when the conversion isn't needed, but always exist so that code can be written in a portable way.
Very little code actually needed updating to use UTF-16, as the vast majority makes no assumptions about the size of TCHAR, nor how FString should be serialized. The main places were the FString archive serialization and the JSON reader/writer, along with some minor fixes to the UTF-8 conversion logic for platforms using a 4-byte TCHAR.
Tests have been added to verify that an FString representing a UTF-32 code point can be losslessly converted to/from UTF-8 and UTF-16, and serialized to/from an archive.
#jira
#rb Steve.Robb, Josh.Adams
#ROBOMERGE-SOURCE: CL 8676728 via CL 8687863
#ROBOMERGE-BOT: (v421-8677696)
[CL 8688048 by jamie dale in Main branch]
Change 8553543 by Wes.Fudala
We now set a fixed value of 24 to CEF screenInfo colorDepth when off screen rendering is enabled.
Change 8235770 by Wes.Fudala
Fix for CEFJsScripting memory stomp and memory alignment errors called out by the stomp memory allocator when running with -stompmalloc on the commandline.
Change 8065597 by Leigh.Swift
BuildPatchServices: Improving ChunkBuildDirectory behaviour when dealing with empty build or builds only containing empty files.
This is now fully supported as part of the generation flow rather than an early detected edge case.
Change 7641628 by Leigh.Swift
BuildPatchServices: Tweaks to serialisation safety.
BuildPatchServices: DiskChunkStore fix for crash when chunkdump serialisation fails due to disk space.
Change 7436869 by Leigh.Swift
BuildPatchServices: Fix regression in BPT package chunks for cloud save improvements now require message pumping.
Change 7326553 by Wes.Fudala
BuildPatchServices: Attempt to restore functionality of the -SkipBuildPatchPrereq commandline. This stopped functioning in CL# 6655502 with the BPS DLC related refactors.
The issue was reported by a number of users that were relying on this commandline as a last resort workaround for prereq install issues.
Change 7323945 by Leigh.Swift
BuildPatchServices: BuildPatchTool: Install time coefficient values exposed by DiffManifests.
This gives an indicative install duration.
The time is not necessarily accurate, but the simulation is a constant and so the value is highly comparable between different builds.
Change 7310352 by Antony.Carter
BuildPatchServices: Adding support for overriding http path for chunk requests. This allows the ability to support signed urls when downloading patch data.
Change 7095282 by Leigh.Swift
BuildPatchServices: Fix regression with manifests that have no core files.
Change 7092198 by leigh.swift
BuildPatchServices: Reuse existing code from FBuildPatchAppManifest::GetChunkShaHash in FBuildPatchManifestSet::GetChunkShaHash.
This fixes an issue where older manifest files that did not ship with chunk sha values in them, can skip sha validation of chunks like pre-DLC launcher did.
Change 6959115 by Wes.Fudala
Added functionality that will optionally expose embedded browser console logs to the client.
Change 6835841 by Leigh.Swift
BuildPatchServices: Rearranging manifest save logic to avoid unnecessary seeking forwards, which avoids an assert when undetected write failures occur.
Change 6684994 by Leigh.Swift
BuildPatchServices: Don't clean empty directories if staging only.
Change 6655502 by Mike.Erickson, Leigh.Swift, Wes.Fudala
BuildPatchServices: Restructuring how installers are configured and make use of manifest files in order to combine multiple actions on an installation directory into one installer.
This resembles a feature set for a better DLC installation experience.
Change 6404031 by Richard.Fawcett
BuildPatchTool: Only append ".manifest" to output filename if output filename has been specified on the command line.
This was causing a manifest file called literally ".manifest" to be output to the clouddir if -OutputFilename was not specified.
Change 6077240 by Wes.Fudala
Execution of browser resource load complete delegate now happens on the main thread.
Change 6076171 by Leigh.Swift
BuildPatchTool: PatchGeneration: ChunkDeltaOptimise: PackageChunks: Improved corrupt output protection against ill timed taskkill, by serialising to temp filename, and then rename on success.
BuildPatchTool: PatchGeneration: Manifest file extension added if not provided, fixing an oversight and inconsistency with other mode behaviours.
BuildPatchTool: Compactify: Only warn when failing to get a file size, if the file still exists. Otherwise log instead.
Change 6049003 by Leigh.Swift
BuildPatchServices: Adding ProcessRequiredDiskSpace to Launcher.Install.Stats which represents how much disk space the install/update process needed to complete.
BuildPatchServices: Adding ProcessAvailableDiskSpace to Launcher.Install.Stats which represents how much disk space was available at the time of checking required disk space.
Change 5915157 by Leigh.Swift
BuildPatchTool: Adding a statistic to diffmanifests for temporary disk space requirement to apply the patch.
Change 5934838 by Leigh.Swift
BuildPatchTool: PackageChunks: Adding support to provide a tagset for the previous build manifest when producing chunkdbs. This allows expanding the chunks saved out to cover tagsets not installed in the previous build.
Change 5838666 by wes.fudala
Browser can now bubble up the state of completed web resource loads.
Change 5689493 by Leigh.Swift
Adding new x86 and x64 MS VC141 CRT redist, version 14.16.27012
Change 5689462 by Leigh.Swift
Fixing process handle leaks on windows.
Core was leaking for getting an application name.
Change 5500917 by Leigh.Swift
BuildPatchTool: Adding new arg DiffAbortThreshold to ChunkDeltaOptimise mode which allows skipping of the operation if the original delta is so large that it would take too long to process, and likely have little benefit.
BuildPatchTool: Switching some Log output to use Display so that it will appear in EC and CMD windows.
Change 5337482 by Leigh.Swift
BuildPatchTool: Fix for DiffManifests mode not accurately representing delta size for tagged install sets.
Change 5261246 by Leigh.Swift
BuildPatchServices: Fix for file download needing to mock response codes for higher layer statistics code which tracks data sizes and speeds.
This is a regression from previous change to correct download failure vs corruption statistics.
Change 5224725 by Leigh.Swift
BuildPatchServices: Fix for delta download of more than 0 bytes when no update is necessary.
BuildPatchServices: Skip requesting delta metafile if no file changes are actually required for a patch.
BuildPatchTool: Reduce unnecessary data produced by BPT ChunkDeltaOptimise mode.
Change 5010941 by Mike.Erickson
BuildPatchServices: Add download scaling based on average speed per request, maximum count, and download health.
Change 5010845 by Wes.Fudala
BuildPatchServices: IDownload refactored to have specific request and response success functions, to make it clearer that a successful request does not mean the response was also good.
BuildPatchServices: Fixed issues with download failures reporting as corruptions.
Change 5000643 by Wes.Hunt
Remove HttpServiceTracker from UE4.
Change 4884381 by Leigh.Swift
BuildPatchTool: Fix for Package Chunks mode hanging when no chunks were required.
Change 4848675 by Justin.Sargent, Leigh.Swift
Speculative fixes for graphics device lost related crash, by adding additional d3d api result checks.
Improved logging for graphics device lost handling.
Improved logging for tracking down common font loading failure resulting in an ensure.
Change 4831134 by Leigh.Swift
BuildPatchTool: Fix for crash in patchgeneration when fast-forward path replays no match.
Change 4801714 by Wes.Fudala
Fix for CEF issue encountered when building using Mac Mojave + XCode10.
Change 4719149 by Leigh.Swift
BuildPatchTool: PatchGeneration mode cyclic data optimisation, reduces SHA calculation requirement counts for cyclic data.
BuildPatchTool: PatchGeneration mode fix for a bug causing non-optimal match insertion idx searching when there are 10k+ matches per scanner.
Change 4680963 by Leigh.Swift
BuildPatchTool: ChunkDeltaOptimise mode is now FeatureLevel upgrade / downgrade aware.
Change 4680947 by Leigh.Swift
BuildPatchTool: Compactify speed improvements for massive network cloud directories.
Change 4656991 by Leigh.Swift
BuildPatchServices: Make sure chunk writer robustly discovers if a chunk fails to save out.
Change 4647815 by Leigh.Swift
Upping the minimum wait time for UdpMessageBeacon thread so that it will not always wait 0ms when network sends are failing, reducing disconnect CPU usage.
Adding configurable tick rate logic to XmppConnectionJingle thread. It will now default to 100Hz max.
Change 4627355 by Michael.Trepka
Fixed a problem with CEF being unable to find locale pak files on Mac for certain language/region combinations
Change 4620800 by Leigh.Swift
Fix for CEF crash when disabling a web window that has not yet got a parent window. There's no need to worry about focus in this case.
Change 4590207 by Leigh.Swift
BuildPatchTool: PackageChunks mode now supports FeatureLevel arg
Change 4590103 by leigh.swift
BuildPatchTool: Adding new mode ChunkDeltaOptimise which reducing the download size when patching between two specific builds in a specific direction.
BuildPatchTool: Updated Enumeration, DiffManifests, Compactify, PackageChunks, and VerifyChunks modes to take account of new delta data.
BuildPatchServices: Installers now have a single shared memory chunk store, which reduces the requirement for booting
Change 4590089 by Leigh.Swift
BuildPatchTool: Adding new mode ChunkDeltaOptimise which reducing the download size when patching between two specific builds in a specific direction.
BuildPatchTool: Updated Enumeration, DiffManifests, Compactify, PackageChunks, and VerifyChunks modes to take account of new delta data.
BuildPatchServices: Installers now have a single shared memory chunk store, which reduces the requirement for booting
Change 4341076 by Leigh.Swift
BuildPatchServices: Making FBuildPatchAppManifest::GetRemovableFiles more robust to handle directories with or without trailing slash.
Change 4331754 by Leigh.Swift
BuildPatchTool: Added support for selecting ChunkWindowSize when generating patches.
BuildPatchTool: Added support for providing the FeatureLevel command-line argument to indicate the data version that should be saved out by patch generation. This warns about defaulting to LatestJson if not provided.
BuildPatchTool: Added support for generating patches with recognition for any chunks with any ChunkWindowSize found in the provided CloudDir.
BuildPatchTool: Added command-line -IgnoreOtherWindowSizes param which if provided, the generation code will only accept chunk matches that are the same as ChunkWindowSize.
BuildPatchServices: Fixes for supporting installations that use any ChunkWindowSize.
BuildPatchServices: New manifest file format to reduce file size, this is now raw compressed binary data.
#lockdown Nick.Penwarden
#rb none
[CL 8675597 by Leigh Swift in Main branch]
Matt.Kuhlenschmidt
#ROBOMERGE-SOURCE: CL 8576345 via CL 8576348 via CL 8615993
#ROBOMERGE-BOT: (v409-8614070)
[CL 8617964 by nick darnell in Main branch]
Matt.Kuhlenschmidt
[FYI] Michael.Noland
#ROBOMERGE-OWNER: nick.darnell
#ROBOMERGE-AUTHOR: nick.darnell
#ROBOMERGE-SOURCE: CL 8548577 via CL 8548824 via CL 8555983
#ROBOMERGE-BOT: (v406-8472469)
[CL 8556177 by nick darnell in Main branch]
- Secondary users now have a faux cursor to control
- Slate now supports per-user tooltips and drag-drop operations
- FSlateUsers are now accessible from outside of FSlateApplication
- Internally enforcing validity of the UserIndex associated with input events within GetOrCreateUser/RegisterNewUser
- ForEachUser provides an FSlateUser& now
- Added the SLATE_SCOPE macro to allow public access to class elements from within the slate module only (external modules see them as protected)
[REVIEW] [at]Nick.Darnell, [at]Nick.Atamas, [at]Matt.Kuhlenschmidt, [at]Chris.Gagnon, [at]Cody.Albert
#rb Nick.Darnell
#ROBOMERGE-SOURCE: CL 8494217 via CL 8501221
#ROBOMERGE-BOT: (v406-8472469)
[CL 8502630 by dan hertzka in Main branch]
Note: Bitmap fonts require FreeType 2.10, and support is compiled out if using an older version.
Font Rendering:
- ApplySizeAndScale now computes the desired pixel size manually (using our desired DPI) rather than calling FT_Set_Char_Size.
- This also includes the font scale, which avoids the call to FT_Set_Transform and lets us remove some workarounds for inconsistent scaling from FreeType.
- For scalable fonts the desired pixel size is passed to FT_Set_Pixel_Sizes.
- For bitmap fonts the desired pixel size is used to find the most suitable strike size (set via FT_Select_Size) and the ratio needed to scale that strike to the desired size (see GetBitmapAtlasScale and GetBitmapRenderScale).
- AppendGlyphFlags has been updated to load color data (when available), and to only exclude bitmap data for fonts that also have scalable data.
- FSlateFontRenderer::GetRenderDataInternal has been updated to handle color bitmap rendering (BGRA, sRGB) in addition to the existing grayscale rendering.
- It will also handle scaling down bitmap font strikes that are larger than the desired size, which saves space in the atlas (see GetBitmapAtlasScale).
- It won't scale up any bitmaps, as that is handled by the transform applied when batching the glyph quads for rendering (see GetBitmapRenderScale).
- Added support for disabling outline and tint for bitmap fonts (in Slate and Canvas).
- Added support for batching both grayscale (8-bit alpha-only) and color (8-bit per-channel BGRA) font quads from a font texture atlas (in RHI, D3D, and OpenGL).
- Removed unused data from FCharacterRenderData.
#jira
#rb Matt.Kuhlenschmidt
#ROBOMERGE-SOURCE: CL 8177432 via CL 8197318
#ROBOMERGE-BOT: (v401-8057353)
[CL 8197520 by jamie dale in Main branch]
