Engine/Editor changes:
- Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries.
- Made the summary sender as agnostics as possible of the keys it sends.
- Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315).
- Fixed concurrent issue when saving the summary sessions on Linux/Mac
- Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency.
- Fixed cases where the same session summary is sent more than once.
- Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen)
- Made adding new properties to the summary easy and private to the implementation.
- Brought the Linux/Mac implementation closer to Windows implementation.
- Reduced memory allocation, especially when the session records a crash.
- Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly.
- Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement
- Reserved the disk space required to store the summary ahead of time to prevent failing later.
- Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes).
- Added unit tests
CrashReportClient changes:
- Created a 'session summary' from the CRC point of view to merge with the Editor summary.
- Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics
- Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary.
- Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things
- Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out)
- Added MonitorSessionDuration summary property to track now long CRC ran.
- Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234
- Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash.
- Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts.
- Added IsProcessingCrash summary property to track when CRC dies processing a crash.
- Added MonitorCrashed summary property to track when CRC exception handler was triggered.
- Added MonitorWasShutdown summary property to track when CRC summary was shutdown
- Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer).
- More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash)
- Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash).
#rb Jamie.Dale, Wes.Hunt, Johan.Berg
#jira UETOOL-3500
#jira UE-114315
[CL 16324612 by Patrick Laflamme in ue5-main branch]
[at]john.nielson [at]todd.eckert
#tests PIE, client, server
#ROBOMERGE-SOURCE: CL 13062778 via CL 13062781 via CL 13062785
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v686-13045012)
[CL 13062792 by wes hunt in Main branch]
#rb Cory.Kolek
#ROBOMERGE-SOURCE: CL 12508127 via CL 12508133 via CL 12508137 via CL 12508141
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v673-12478461)
[CL 12508147 by federico oro in Main branch]
- Added AppId, AppVersion and UserId as FEditorAnalyticSession members and updated FEditorSessionSummaryWriter to write them.
- Changed FEngineAnalytics::GetProvider() to return IAnalyticsProviderET rather than IAnalyticsProvider.
- Updated FEditorSessionSummarySender to embed the AppId/AppVersion/UserId as attribute when sending the summary event and also configured the AnalyticsProvider to use the SessionID/AppID/AppVersion/UserID of the event rather than the current one.
- Added a function to return the current IAnalyticProviderET configuration.
#rb Wes.Hunt
Edigrated 11445334 and 11459406.
#ROBOMERGE-SOURCE: CL 11512055 in //UE4/Release-4.25/...
#ROBOMERGE-BOT: RELEASE (Release-4.25 -> Release-4.25Plus) (v654-11333218)
[CL 11512070 by patrick laflamme in 4.25-Plus branch]
#rnx
#rb none
#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869210 via CL 10869511 via CL 10869900
#ROBOMERGE-BOT: (v613-10869866)
[CL 10870549 by ryan durand in Main branch]
Fix up nearby cases where ESearchCase::CaseSensitive should have been used
#jira
#rnx
#rb
#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 10309793 via CL 10309818
#ROBOMERGE-BOT: (v593-10286020)
[CL 10309932 by marc audy in Main branch]
Change 8553543 by Wes.Fudala
We now set a fixed value of 24 to CEF screenInfo colorDepth when off screen rendering is enabled.
Change 8235770 by Wes.Fudala
Fix for CEFJsScripting memory stomp and memory alignment errors called out by the stomp memory allocator when running with -stompmalloc on the commandline.
Change 8065597 by Leigh.Swift
BuildPatchServices: Improving ChunkBuildDirectory behaviour when dealing with empty build or builds only containing empty files.
This is now fully supported as part of the generation flow rather than an early detected edge case.
Change 7641628 by Leigh.Swift
BuildPatchServices: Tweaks to serialisation safety.
BuildPatchServices: DiskChunkStore fix for crash when chunkdump serialisation fails due to disk space.
Change 7436869 by Leigh.Swift
BuildPatchServices: Fix regression in BPT package chunks for cloud save improvements now require message pumping.
Change 7326553 by Wes.Fudala
BuildPatchServices: Attempt to restore functionality of the -SkipBuildPatchPrereq commandline. This stopped functioning in CL# 6655502 with the BPS DLC related refactors.
The issue was reported by a number of users that were relying on this commandline as a last resort workaround for prereq install issues.
Change 7323945 by Leigh.Swift
BuildPatchServices: BuildPatchTool: Install time coefficient values exposed by DiffManifests.
This gives an indicative install duration.
The time is not necessarily accurate, but the simulation is a constant and so the value is highly comparable between different builds.
Change 7310352 by Antony.Carter
BuildPatchServices: Adding support for overriding http path for chunk requests. This allows the ability to support signed urls when downloading patch data.
Change 7095282 by Leigh.Swift
BuildPatchServices: Fix regression with manifests that have no core files.
Change 7092198 by leigh.swift
BuildPatchServices: Reuse existing code from FBuildPatchAppManifest::GetChunkShaHash in FBuildPatchManifestSet::GetChunkShaHash.
This fixes an issue where older manifest files that did not ship with chunk sha values in them, can skip sha validation of chunks like pre-DLC launcher did.
Change 6959115 by Wes.Fudala
Added functionality that will optionally expose embedded browser console logs to the client.
Change 6835841 by Leigh.Swift
BuildPatchServices: Rearranging manifest save logic to avoid unnecessary seeking forwards, which avoids an assert when undetected write failures occur.
Change 6684994 by Leigh.Swift
BuildPatchServices: Don't clean empty directories if staging only.
Change 6655502 by Mike.Erickson, Leigh.Swift, Wes.Fudala
BuildPatchServices: Restructuring how installers are configured and make use of manifest files in order to combine multiple actions on an installation directory into one installer.
This resembles a feature set for a better DLC installation experience.
Change 6404031 by Richard.Fawcett
BuildPatchTool: Only append ".manifest" to output filename if output filename has been specified on the command line.
This was causing a manifest file called literally ".manifest" to be output to the clouddir if -OutputFilename was not specified.
Change 6077240 by Wes.Fudala
Execution of browser resource load complete delegate now happens on the main thread.
Change 6076171 by Leigh.Swift
BuildPatchTool: PatchGeneration: ChunkDeltaOptimise: PackageChunks: Improved corrupt output protection against ill timed taskkill, by serialising to temp filename, and then rename on success.
BuildPatchTool: PatchGeneration: Manifest file extension added if not provided, fixing an oversight and inconsistency with other mode behaviours.
BuildPatchTool: Compactify: Only warn when failing to get a file size, if the file still exists. Otherwise log instead.
Change 6049003 by Leigh.Swift
BuildPatchServices: Adding ProcessRequiredDiskSpace to Launcher.Install.Stats which represents how much disk space the install/update process needed to complete.
BuildPatchServices: Adding ProcessAvailableDiskSpace to Launcher.Install.Stats which represents how much disk space was available at the time of checking required disk space.
Change 5915157 by Leigh.Swift
BuildPatchTool: Adding a statistic to diffmanifests for temporary disk space requirement to apply the patch.
Change 5934838 by Leigh.Swift
BuildPatchTool: PackageChunks: Adding support to provide a tagset for the previous build manifest when producing chunkdbs. This allows expanding the chunks saved out to cover tagsets not installed in the previous build.
Change 5838666 by wes.fudala
Browser can now bubble up the state of completed web resource loads.
Change 5689493 by Leigh.Swift
Adding new x86 and x64 MS VC141 CRT redist, version 14.16.27012
Change 5689462 by Leigh.Swift
Fixing process handle leaks on windows.
Core was leaking for getting an application name.
Change 5500917 by Leigh.Swift
BuildPatchTool: Adding new arg DiffAbortThreshold to ChunkDeltaOptimise mode which allows skipping of the operation if the original delta is so large that it would take too long to process, and likely have little benefit.
BuildPatchTool: Switching some Log output to use Display so that it will appear in EC and CMD windows.
Change 5337482 by Leigh.Swift
BuildPatchTool: Fix for DiffManifests mode not accurately representing delta size for tagged install sets.
Change 5261246 by Leigh.Swift
BuildPatchServices: Fix for file download needing to mock response codes for higher layer statistics code which tracks data sizes and speeds.
This is a regression from previous change to correct download failure vs corruption statistics.
Change 5224725 by Leigh.Swift
BuildPatchServices: Fix for delta download of more than 0 bytes when no update is necessary.
BuildPatchServices: Skip requesting delta metafile if no file changes are actually required for a patch.
BuildPatchTool: Reduce unnecessary data produced by BPT ChunkDeltaOptimise mode.
Change 5010941 by Mike.Erickson
BuildPatchServices: Add download scaling based on average speed per request, maximum count, and download health.
Change 5010845 by Wes.Fudala
BuildPatchServices: IDownload refactored to have specific request and response success functions, to make it clearer that a successful request does not mean the response was also good.
BuildPatchServices: Fixed issues with download failures reporting as corruptions.
Change 5000643 by Wes.Hunt
Remove HttpServiceTracker from UE4.
Change 4884381 by Leigh.Swift
BuildPatchTool: Fix for Package Chunks mode hanging when no chunks were required.
Change 4848675 by Justin.Sargent, Leigh.Swift
Speculative fixes for graphics device lost related crash, by adding additional d3d api result checks.
Improved logging for graphics device lost handling.
Improved logging for tracking down common font loading failure resulting in an ensure.
Change 4831134 by Leigh.Swift
BuildPatchTool: Fix for crash in patchgeneration when fast-forward path replays no match.
Change 4801714 by Wes.Fudala
Fix for CEF issue encountered when building using Mac Mojave + XCode10.
Change 4719149 by Leigh.Swift
BuildPatchTool: PatchGeneration mode cyclic data optimisation, reduces SHA calculation requirement counts for cyclic data.
BuildPatchTool: PatchGeneration mode fix for a bug causing non-optimal match insertion idx searching when there are 10k+ matches per scanner.
Change 4680963 by Leigh.Swift
BuildPatchTool: ChunkDeltaOptimise mode is now FeatureLevel upgrade / downgrade aware.
Change 4680947 by Leigh.Swift
BuildPatchTool: Compactify speed improvements for massive network cloud directories.
Change 4656991 by Leigh.Swift
BuildPatchServices: Make sure chunk writer robustly discovers if a chunk fails to save out.
Change 4647815 by Leigh.Swift
Upping the minimum wait time for UdpMessageBeacon thread so that it will not always wait 0ms when network sends are failing, reducing disconnect CPU usage.
Adding configurable tick rate logic to XmppConnectionJingle thread. It will now default to 100Hz max.
Change 4627355 by Michael.Trepka
Fixed a problem with CEF being unable to find locale pak files on Mac for certain language/region combinations
Change 4620800 by Leigh.Swift
Fix for CEF crash when disabling a web window that has not yet got a parent window. There's no need to worry about focus in this case.
Change 4590207 by Leigh.Swift
BuildPatchTool: PackageChunks mode now supports FeatureLevel arg
Change 4590103 by leigh.swift
BuildPatchTool: Adding new mode ChunkDeltaOptimise which reducing the download size when patching between two specific builds in a specific direction.
BuildPatchTool: Updated Enumeration, DiffManifests, Compactify, PackageChunks, and VerifyChunks modes to take account of new delta data.
BuildPatchServices: Installers now have a single shared memory chunk store, which reduces the requirement for booting
Change 4590089 by Leigh.Swift
BuildPatchTool: Adding new mode ChunkDeltaOptimise which reducing the download size when patching between two specific builds in a specific direction.
BuildPatchTool: Updated Enumeration, DiffManifests, Compactify, PackageChunks, and VerifyChunks modes to take account of new delta data.
BuildPatchServices: Installers now have a single shared memory chunk store, which reduces the requirement for booting
Change 4341076 by Leigh.Swift
BuildPatchServices: Making FBuildPatchAppManifest::GetRemovableFiles more robust to handle directories with or without trailing slash.
Change 4331754 by Leigh.Swift
BuildPatchTool: Added support for selecting ChunkWindowSize when generating patches.
BuildPatchTool: Added support for providing the FeatureLevel command-line argument to indicate the data version that should be saved out by patch generation. This warns about defaulting to LatestJson if not provided.
BuildPatchTool: Added support for generating patches with recognition for any chunks with any ChunkWindowSize found in the provided CloudDir.
BuildPatchTool: Added command-line -IgnoreOtherWindowSizes param which if provided, the generation code will only accept chunk matches that are the same as ChunkWindowSize.
BuildPatchServices: Fixes for supporting installations that use any ChunkWindowSize.
BuildPatchServices: New manifest file format to reduce file size, this is now raw compressed binary data.
#lockdown Nick.Penwarden
#rb none
[CL 8675597 by Leigh Swift in Main branch]
[at]paul.moore
#tests run a local server, wait for hotfix, verify that version replacement works and no version change is made.
#rnx
#ROBOMERGE-SOURCE: CL 6607969 via CL 6608412 via CL 6609654
#ROBOMERGE-BOT: (v358-6608238)
[CL 6609848 by wes hunt in Main branch]
- Replace format of app version sent in ET analytics to: Season.Minor.Hotfix - %VERSION%
[REVIEW]
[at]wes.hunt
#ROBOMERGE-SOURCE: CL 6518009 via CL 6518019 via CL 6518034 via CL 6523213
[CL 6523249 by paul moore in Main branch]
[FYI] david.nikdel
#tests This is split from another larger change to keep the engine changes separate.
#ROBOMERGE-SOURCE: CL 6245913 via CL 6246221
[CL 6246235 by wes hunt in Main branch]
Also add an overload for FStrings to allow us to avoid copies in that specific case, since we can't use a universal qualifier in the primary template, which caused unnecessary string copies to be made in almost all the code that added attributes.
[FYI] dan.hertzka, bob.tellez, matt.loesby, ben.marsh
#rnx
#ROBOMERGE-SOURCE: CL 6046823 via CL 6047228
[CL 6047267 by wes hunt in Main branch]
