Files
UnrealEngineUWP/Engine/Source/Programs/CrashReportClient/Private/CrashReportAnalyticsSessionSummary.cpp

738 lines
28 KiB
C++
Raw Normal View History

Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
// Copyright Epic Games, Inc. All Rights Reserved.
#include "CrashReportAnalyticsSessionSummary.h"
#include "AnalyticsPropertyStore.h"
#include "AnalyticsSessionSummaryManager.h"
#include "AnalyticsSessionSummarySender.h"
#include "IAnalyticsProviderET.h"
#include "CrashReportAnalytics.h"
#include "Containers/Map.h"
#include "GenericPlatform/GenericPlatformMath.h"
#include "GenericPlatform/GenericPlatformCrashContext.h"
#include "HAL/PlatformTime.h"
#include "HAL/PlatformProcess.h"
#include "HAL/Thread.h"
#include "Internationalization/Regex.h"
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
#include "Logging/LogMacros.h"
#include "Misc/EngineVersion.h"
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
#include "Misc/Paths.h"
#include "Misc/ScopeLock.h"
#include "Misc/CoreDelegates.h"
#include "Misc/OutputDeviceRedirector.h"
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
#include "Templates/UnrealTemplate.h"
DEFINE_LOG_CATEGORY_STATIC(LogCrashReportClientDiagnostics, Log, All)
#if PLATFORM_WINDOWS
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
#include "Windows/AllowWindowsPlatformTypes.h"
#include "Windows.h"
/** Handle windows messages. */
LRESULT CALLBACK CrashReportAnalyticsSessionSummaryWindowProc(HWND Hwnd, UINT Msg, WPARAM wParam, LPARAM lParam)
{
// wParam is true if the user session is going away (and CRC is going to die)
if (Msg == WM_ENDSESSION && wParam == TRUE)
{
FCrashReportAnalyticsSessionSummary::Get().OnUserLoggingOut();
}
else if (Msg == WM_CLOSE)
{
FCrashReportAnalyticsSessionSummary::Get().OnQuitSignal();
}
return DefWindowProc(Hwnd, Msg, wParam, lParam);
}
FProcHandle OpenProcessForMonitoring(uint32 pid)
{
// Until a crash occurs, for security reasons, restrict CRC accesses on the remote process.
return FProcHandle(::OpenProcess(PROCESS_DUP_HANDLE | PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_TERMINATE | SYNCHRONIZE, 0, pid));
}
#include "Windows/HideWindowsPlatformTypes.h"
#else // PLATFORM_WINDOWS
FProcHandle OpenProcessForMonitoring(uint32 pid)
{
return FPlatformProcess::OpenProcess(pid);
}
#endif // PLATFORM_WINDOWS
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
namespace CrcAnalyticsProperties
{
// NOTE: Update this when you add/remove/change key behavior. That's useful to track how one changes affects metrics in-dev where users don't always have an engine versions.
// - V3 -> Windows optimization for stall/ensure -> The engine only captures the responsible thread so CRC walks 1 thread rather than all threads.
// - V4 -> Stripped ensure callstack from the ensure error message to remove noise in the diagnostic log.
// - V5 -> Measured time to stack-walk, gather files and addded stall count.
// - V6 -> Added MonitorTickCount, MonitorQueryingPipe and App/Death log.
// - V7 -> Removed MonitorQueryingPipe. It was added to detect if CRC crashed while reading the pipe. Data showed that wasn't the case.
constexpr uint32 CrcAnalyticsSummaryVersion = 7;
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
/** The exit code of the monitored application. */
static const TAnalyticsProperty<int32> MonitoredAppExitCode(TEXT("ExitCode"));
/** Track when CRC detected the death of the monitored app. */
static const TAnalyticsProperty<FDateTime> MonitoredAppDeathTimestamp(TEXT("DeathTimestamp"));
/** CRC engine version. In-dev, people don't always recompile CRC and we get disparity between the monitored app and CRC version. */
static const TAnalyticsProperty<FString> EngineVersion(TEXT("MonitorEngineVersion"));
/** The version number of the key/set used by CRC. */
static const TAnalyticsProperty<uint32> SummaryVersionNumber(TEXT("MonitorSummaryVersion"));
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
/** The CRC startup timestamp. */
static const TAnalyticsProperty<FDateTime> StartupTimestamp(TEXT("MonitorStartupTimestamp"));
/** The CRC timestamp. */
static const TAnalyticsProperty<FDateTime> Timestamp(TEXT("MonitorTimestamp"));
/** Number of time CRC analytic thread ticked. */
static const TAnalyticsProperty<uint32> TickCount(TEXT("MonitorTickCount"));
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
/** If CRC raised an exception that was captured by SEH, this is the exception code. */
static const TAnalyticsProperty<int32> ExceptCode(TEXT("MonitorExceptCode"));
/** If CRC is about to close because the system sent a quit signal. */
static const TAnalyticsProperty<bool> QuitSignalRecv(TEXT("MonitorQuitSignalRecv"));
/** The CRC diagnostic logs. */
static const TAnalyticsProperty<FString> DiagnosticLogs(TEXT("MonitorLog"));
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
/** The CRC session duration in seconds. */
static const TAnalyticsProperty<int32> SessionDurationSecs(TEXT("MonitorSessionDuration"));
/** The battery level, if known. */
static const TAnalyticsProperty<uint32> BatteryLevel(TEXT("MonitorBatteryLevel"));
/** If the system is connected to AC, if know */
static const TAnalyticsProperty<bool> IsOnACPower(TEXT("MonitorOnACPower"));
/** Whether CRC is reporting a crash. */
static const TAnalyticsProperty<bool> IsReportingCrash(TEXT("MonitorIsReportingCrash"));
/** Whether CRC is collecting crash artifacts. */
static const TAnalyticsProperty<bool> IsCollectingCrash(TEXT("MonitorIsCollectingCrash"));
/** Whether CRC is processing a crash. */
static const TAnalyticsProperty<bool> IsProcessingCrash(TEXT("MonitorIsProcessingCrash"));
/** If CRC is about to be killed because the user is logging out (system shutdown/reboot). */
static const TAnalyticsProperty<bool> UserIsLoggingOut(TEXT("MonitorLoggingOut"));
/** Whether CRC crashed. */
static const TAnalyticsProperty<bool> IsCrashing(TEXT("MonitorCrashed"));
/** Whether CRC was shutdown. */
static const TAnalyticsProperty<bool> WasShutdown(TEXT("MonitorWasShutdown"));
/** Number of crash event passed to CRC. (Ensure, Assert, Crash, etc). */
static const TAnalyticsProperty<uint32> ReportCount(TEXT("MonitorReportCount"));
/** Number of ensures handled by CRC. */
static const TAnalyticsProperty<uint32> EnsureCount(TEXT("MonitorEnsureCount"));
/** Number of assert handled by CRC.*/
static const TAnalyticsProperty<uint32> AssertCount(TEXT("MonitorAssertCount"));
/** Number of stalls handed by CRC. */
static const TAnalyticsProperty<uint32> StallCount(TEXT("MonitorStallCount"));
/** The worst unattended report time measured (if any). The user is not involved, so it measure how fast CRC can process a crash, especially ensures and stalls. */
static const TAnalyticsProperty<float> LonguestUnattendedReportSecs(TEXT("MonitorLongestUnattendedReportSecs"));
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
}
namespace CrashReportClientUtils
{
/** Flush the store peridically. */
static const FTimespan PropertyStoreFlushPeriod = FTimespan::FromSeconds(10);
/** Tick periodically to poll new information. */
static const FTimespan TickPeriod = FTimespan::FromSeconds(0.5);
/** Maximum length of the diagnostic log. */
static constexpr int32 MaxDiagnosticLogLen = 8 * 1024;
#if PLATFORM_WINDOWS && CRASH_REPORT_WITH_MTBF
HWND Hwnd = NULL;
/** Create a hidden Windows to intercept WM_ messages, especially the WM_ENDSESSION. */
void InitPlatformSpecific()
{
// Register the window class.
const wchar_t CLASS_NAME[] = L"CRC Analytics Session Window Message Interceptor";
WNDCLASS wc = { };
wc.lpfnWndProc = CrashReportAnalyticsSessionSummaryWindowProc;
wc.hInstance = hInstance;
wc.lpszClassName = CLASS_NAME;
RegisterClass(&wc);
// Create a window to capture WM_ENDSESSION message (so that we can detect when CRC fails because the user is logging off/shutting down/restarting)
Hwnd = CreateWindowEx(
0, // Optional window styles.
CLASS_NAME, // Window class
L"CRC Message Loop Wnd", // Window text
WS_OVERLAPPEDWINDOW, // Window style
CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, // Size and position
NULL, // Parent window
NULL, // Menu
hInstance, // Instance handle
NULL // Additional application data
);
}
/** Pump the message from the hidden Windows. */
void TickPlatformSpecific()
{
if (Hwnd != NULL)
{
// Pump the messages.
MSG Msg = { };
while (::PeekMessage(&Msg, NULL, 0, 0, PM_REMOVE))
{
::TranslateMessage(&Msg);
::DispatchMessage(&Msg);
}
}
}
bool GetPowerStatus(TOptional<bool>& OutACPowerConnected, TOptional<uint32>& OutBatteryPct)
{
bool bAvailable = false;
SYSTEM_POWER_STATUS Status;
if (GetSystemPowerStatus(&Status))
{
switch (Status.ACLineStatus)
{
case 0: // AC Offline
OutACPowerConnected.Emplace(false);
bAvailable = true;
break;
case 1: // AC Online
OutACPowerConnected.Emplace(true);
bAvailable = true;
break;
default: // Unknown
break;
}
if (Status.BatteryLifePercent != 255) // Unknown
{
OutBatteryPct.Emplace(Status.BatteryLifePercent);
bAvailable = true;
}
}
return bAvailable;
}
#else
void InitPlatformSpecific(){}
void TickPlatformSpecific(){}
bool GetPowerStatus(TOptional<bool>& OutACPowerConnected, TOptional<uint32>& OutBatteryPct) { return false; }
#endif
} // CrashReportClientUtils
/**
* Augments the default summary senders to perform a short analytis to detect if this is the session ended abnormally. The sender searches
* for very specific keys published by the engine. Remember that CRC merges its summary with the monitored process summary (the Editor), so
* it gets to see what was recorded (or not) by the monitored process.
*/
class FCrashReportClientAnalyticsSessionSummarySender : public FAnalyticsSessionSummarySender
{
public:
FCrashReportClientAnalyticsSessionSummarySender(IAnalyticsProviderET& Provider)
: FAnalyticsSessionSummarySender(Provider)
{
}
virtual bool SendSessionSummary(const FString& UserId, const FString& AppId, const FString& AppVersion, const FString& SessionId, const TArray<FAnalyticsEventAttribute>& Properties) override
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
{
// CRC should only send one session (its own), but do a reset in case this convention changes.
bAbnormalShutdown = false;
bUserLoggingOut = false;
// Analyze the report to be sent and try to figure out if this is an abnormal shutdown. They keys are taken from the engine analytics session summary. To prevent dependencies
// between CRC and Engine analytics, we duplicate the keys here.
if (const FAnalyticsEventAttribute* ShutdownTypeCode = Properties.FindByPredicate([](const FAnalyticsEventAttribute& Candidate) { return Candidate.GetName() == FAnalyticsSessionSummaryManager::ShutdownTypeCodeProperty.Key; }))
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
{
bAbnormalShutdown = (ShutdownTypeCode->GetValue() == LexToString((int32)EAnalyticsSessionShutdownType::Abnormal));
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
if (bAbnormalShutdown)
{
if (const FAnalyticsEventAttribute* UserLoggingOut = Properties.FindByPredicate([](const FAnalyticsEventAttribute& Candidate) { return Candidate.GetName() == FAnalyticsSessionSummaryManager::IsUserLoggingOutProperty.Key; }))
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
{
bUserLoggingOut = (UserLoggingOut->GetValue() == LexToString(true));
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
}
}
}
// Send the report unmodified.
return FAnalyticsSessionSummarySender::SendSessionSummary(UserId, AppId, AppVersion, SessionId, Properties);
}
/** Returns whether the last session sent was abnormally terminated. */
bool IsAbnormalShutdown() const
{
return bAbnormalShutdown && !bUserLoggingOut;
}
private:
bool bAbnormalShutdown = false;
bool bUserLoggingOut = false;
};
FCrashReportAnalyticsSessionSummary::FCrashReportAnalyticsSessionSummary()
: SessionStartTimeSecs(FPlatformTime::Seconds())
, bMonitoredAppDeathRecorded(false)
, bShutdown(false)
{
if (IsEnabled())
{
CrashReportClientUtils::InitPlatformSpecific();
// Reserve the memory for the log string.
DiagnosticLog.Reset(CrashReportClientUtils::MaxDiagnosticLogLen);
DiagnosticLog.Append(FString::Printf(TEXT("CRC/Init:%s"), *FDateTime::UtcNow().ToString()));
}
}
FCrashReportAnalyticsSessionSummary& FCrashReportAnalyticsSessionSummary::FCrashReportAnalyticsSessionSummary::Get()
{
static FCrashReportAnalyticsSessionSummary Instance;
return Instance;
}
void FCrashReportAnalyticsSessionSummary::Initialize(const FString& ProcessGroupId, uint32 ForProcessId)
{
if (IsEnabled() && !SessionSummaryManager && !ProcessGroupId.IsEmpty())
{
SessionSummaryManager = MakeUnique<FAnalyticsSessionSummaryManager>(TEXT("CrashReportClient"), ProcessGroupId, ForProcessId);
if (SessionSummaryManager)
{
constexpr uint32 ReservedFileCapacity = CrashReportClientUtils::MaxDiagnosticLogLen + (4 * 1024);
PropertyStore = SessionSummaryManager->MakeStore(ReservedFileCapacity);
if (PropertyStore)
{
FCoreDelegates::ApplicationWillTerminateDelegate.AddRaw(this, &FCrashReportAnalyticsSessionSummary::OnApplicationWillTerminate);
FCoreDelegates::OnHandleSystemError.AddRaw(this, &FCrashReportAnalyticsSessionSummary::OnHandleSystemError);
CrcAnalyticsProperties::EngineVersion.Set(PropertyStore.Get(), FEngineVersion::Current().ToString(EVersionComponent::Changelist));
CrcAnalyticsProperties::SummaryVersionNumber.Set(PropertyStore.Get(), CrcAnalyticsProperties::CrcAnalyticsSummaryVersion);
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
CrcAnalyticsProperties::StartupTimestamp.Set(PropertyStore.Get(), FDateTime::UtcNow());
CrcAnalyticsProperties::Timestamp.Set(PropertyStore.Get(), FDateTime::UtcNow());
CrcAnalyticsProperties::TickCount.Set(PropertyStore.Get(), 0);
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
CrcAnalyticsProperties::SessionDurationSecs.Set(PropertyStore.Get(), FMath::FloorToInt(static_cast<float>(FPlatformTime::Seconds() - SessionStartTimeSecs)));
CrcAnalyticsProperties::DiagnosticLogs.Set(PropertyStore.Get(), DiagnosticLog, CrashReportClientUtils::MaxDiagnosticLogLen);
CrcAnalyticsProperties::IsReportingCrash.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::IsCollectingCrash.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::IsProcessingCrash.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::UserIsLoggingOut.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::IsCrashing.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::WasShutdown.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::QuitSignalRecv.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::ReportCount.Set(PropertyStore.Get(), 0);
CrcAnalyticsProperties::EnsureCount.Set(PropertyStore.Get(), 0);
CrcAnalyticsProperties::AssertCount.Set(PropertyStore.Get(), 0);
CrcAnalyticsProperties::StallCount.Set(PropertyStore.Get(), 0);
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
UpdatePowerStatus();
Flush();
GLog->AddOutputDevice(this);
// CRC main thread might be busy processing a crash, so use a background thread to record important events that could otherwise be missed.
AnalyticsThread = MakeUnique<FThread>(TEXT("AnalyticsMonitorThread"), [this, ForProcessId]()
{
// Try to open the process.
FProcHandle MonitoredProcessHandle = OpenProcessForMonitoring(ForProcessId);
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
if (!MonitoredProcessHandle.IsValid())
{
LogEvent(TEXT("CRC/OpenProcessFailed"));
return;
}
CrashReportClientUtils::InitPlatformSpecific();
double NextFlushTimeSecs = FPlatformTime::Seconds();
bool bFlushedLowBattery = false;
LogEvent(TEXT("CRC/Monitoring")); // About to enter the loop.
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
while (!bShutdown)
{
CrcAnalyticsProperties::TickCount.Update(PropertyStore.Get(), [](uint32& Actual) { ++Actual; return true; });
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
CrashReportClientUtils::TickPlatformSpecific();
// Monitor the power level.
bool bShouldFlush = UpdatePowerStatus();
if (!FPlatformProcess::IsProcRunning(MonitoredProcessHandle))
{
OnMonitoredAppDeath(MonitoredProcessHandle);
break;
}
else if (FPlatformTime::Seconds() > NextFlushTimeSecs || bShouldFlush)
{
// Flush to timestamp the session.
Flush();
NextFlushTimeSecs += CrashReportClientUtils::PropertyStoreFlushPeriod.GetTotalSeconds();
}
// Throttle the thread.
FPlatformProcess::Sleep(CrashReportClientUtils::TickPeriod.GetTotalSeconds());
}
FPlatformProcess::CloseProc(MonitoredProcessHandle);
});
}
}
}
}
bool FCrashReportAnalyticsSessionSummary::IsValid() const
{
return IsEnabled() && PropertyStore.IsValid();
}
void FCrashReportAnalyticsSessionSummary::Shutdown(IAnalyticsProviderET* AnalyticsProvider, TFunction<void()> HandleAbnormalShutdownFn)
{
if (!IsValid())
{
return;
}
if (AnalyticsThread)
{
bShutdown = true;
AnalyticsThread->Join();
AnalyticsThread.Reset();
}
CrcAnalyticsProperties::WasShutdown.Set(PropertyStore.Get(), true);
LogEvent(FString::Printf(TEXT("CRC/Shutdown:%s:%.1fs"), *FDateTime::UtcNow().ToString(), FPlatformTime::Seconds() - SessionStartTimeSecs));
// Unregister from the core.
FCoreDelegates::ApplicationWillTerminateDelegate.RemoveAll(this);
FCoreDelegates::OnHandleSystemError.RemoveAll(this);
GLog->RemoveOutputDevice(this);
// Flush and close CRC analytics summary.
Flush();
PropertyStore.Reset();
// If CRC is allowed to send summary report on behalf of the monitored application.
if (AnalyticsProvider)
{
// Set a summary sender that will intercept the set of properties and perform an analysis to detect an abnormal shutdown.
TSharedPtr<FCrashReportClientAnalyticsSessionSummarySender> SummarySender = MakeShared<FCrashReportClientAnalyticsSessionSummarySender>(*AnalyticsProvider);
SessionSummaryManager->SetSender(SummarySender);
// Merge the principal process summary with CRC process summary and sends the session summary if CRC is the last process to exit.
SessionSummaryManager->Shutdown();
// If the summary sender detected an abnormal shutdown by checking the summary properties, report it.
if (SummarySender->IsAbnormalShutdown() && HandleAbnormalShutdownFn)
{
HandleAbnormalShutdownFn();
}
}
else // Discard CRC summary.
{
SessionSummaryManager->Shutdown(/*bDiscard*/true);
}
SessionSummaryManager.Reset();
}
void FCrashReportAnalyticsSessionSummary::Serialize(const TCHAR* V, ELogVerbosity::Type Verbosity, const FName& Category, const double Time)
{
Serialize(V, Verbosity, Category);
}
void FCrashReportAnalyticsSessionSummary::Serialize(const TCHAR* V, ELogVerbosity::Type Verbosity, const FName& Category)
{
// Log the errors, especially the failed 'check()' with the callstack/message.
if (Verbosity == ELogVerbosity::Error || Verbosity == ELogVerbosity::Fatal)
{
// Log but don't forward to UE logging system. The log is already originate from the logging system.
LogEvent(TEXT("CRC/Error"), /*bForwardToUELog*/false);
LogEvent(V, /*bForwardToUELog*/false);
}
}
bool FCrashReportAnalyticsSessionSummary::CanBeUsedOnAnyThread() const
{
return true;
}
bool FCrashReportAnalyticsSessionSummary::CanBeUsedOnMultipleThreads() const
{
return true;
}
void FCrashReportAnalyticsSessionSummary::LogEvent(const FString& Event)
{
LogEvent(*Event);
}
void FCrashReportAnalyticsSessionSummary::LogEvent(const TCHAR* Event, bool bForwardToUELog)
{
if (IsValid())
{
FScopeLock ScopedLock(&LoggerLock);
TGuardValue<bool> ReentrantGuard(bLoggerReentrantGuard, true);
if (*ReentrantGuard) // Read the old value.
{
return; // Prevent renentrant logging.
}
AppendLog(Event);
}
// Prevent error logs coming from the logging system to be duplicated.
if (bForwardToUELog)
{
UE_LOG(LogCrashReportClientDiagnostics, Log, TEXT("%s"), Event);
}
}
void FCrashReportAnalyticsSessionSummary::AppendLog(const TCHAR* Event)
{
// Add the separator if some text is already logged.
if (DiagnosticLog.Len())
{
DiagnosticLog.Append(TEXT("|"));
}
// Rotate the log if it gets too long.
int32 FreeLen = CrashReportClientUtils::MaxDiagnosticLogLen - DiagnosticLog.Len();
int32 EventLen = FCString::Strlen(Event);
if (EventLen > FreeLen)
{
if (EventLen > CrashReportClientUtils::MaxDiagnosticLogLen)
{
DiagnosticLog.Reset(CrashReportClientUtils::MaxDiagnosticLogLen);
EventLen = CrashReportClientUtils::MaxDiagnosticLogLen;
}
else
{
DiagnosticLog.RemoveAt(0, EventLen - FreeLen, /*bAllowShrinking*/false); // Free space, remove the chars from the oldest events (in front).
}
}
// Append the log entry and dump the log to the file.
DiagnosticLog.AppendChars(Event, EventLen);
// Update the diagnostic field into the session summary store.
CrcAnalyticsProperties::DiagnosticLogs.Set(PropertyStore.Get(), DiagnosticLog);
// Flush the store.
Flush();
}
void FCrashReportAnalyticsSessionSummary::OnMonitoredAppDeath(FProcHandle& Handle)
{
// The first thread to exchange successfull is allowed to update. No need to update twice for the same monitored process.
bool Expected = false;
if (IsValid() && bMonitoredAppDeathRecorded.compare_exchange_strong(Expected, true))
{
CrcAnalyticsProperties::MonitoredAppDeathTimestamp.Set(PropertyStore.Get(), FDateTime::UtcNow());
LogEvent(TEXT("App/Death"));
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
int32 ExitCode;
if (Handle.IsValid() && FPlatformProcess::GetProcReturnCode(Handle, &ExitCode))
{
CrcAnalyticsProperties::MonitoredAppExitCode.Set(PropertyStore.Get(), ExitCode);
LogEvent(FString::Printf(TEXT("App/ExitCode:%d"), ExitCode));
}
else
{
CrcAnalyticsProperties::MonitoredAppExitCode.Set(PropertyStore.Get(), ECrashExitCodes::MonitoredApplicationExitCodeNotAvailable);
LogEvent(TEXT("App/ExitCode:N/A"));
}
Flush();
}
}
void FCrashReportAnalyticsSessionSummary::OnUserLoggingOut()
{
if (IsValid())
{
// The user is logging out and CRC is going to die.
CrcAnalyticsProperties::UserIsLoggingOut.Set(PropertyStore.Get(), true);
// Log the event (this also flush the session).
LogEvent(TEXT("CRC/EndSession"));
}
}
void FCrashReportAnalyticsSessionSummary::OnQuitSignal()
{
// The system has requested the app to close. (Like if the user gently kills the application)
CrcAnalyticsProperties::QuitSignalRecv.Set(PropertyStore.Get(), true);
// Log the event (this also flush the session).
LogEvent(TEXT("CRC/QuitSignal"));
}
void FCrashReportAnalyticsSessionSummary::OnCrcCrashing(int32 ExceptCode)
{
if (IsValid())
{
CrcAnalyticsProperties::IsCrashing.Set(PropertyStore.Get(), true);
CrcAnalyticsProperties::ExceptCode.Set(PropertyStore.Get(), ExceptCode);
TCHAR CrashEventLog[64];
FCString::Sprintf(CrashEventLog, TEXT("CRC/Crash:%d"), ExceptCode);
LogEvent(CrashEventLog); // This also flush the session.
}
}
void FCrashReportAnalyticsSessionSummary::Flush()
{
if (IsValid())
{
// Update the session progression.
CrcAnalyticsProperties::Timestamp.Set(PropertyStore.Get(), FDateTime::UtcNow(),
[](const FDateTime* Actual, const FDateTime& Proposed) { return Proposed > *Actual; });
CrcAnalyticsProperties::SessionDurationSecs.Set(PropertyStore.Get(), FMath::FloorToInt(FPlatformTime::Seconds() - SessionStartTimeSecs),
[](const int32* Actual, const int32& Proposed) { return Proposed > *Actual; });
// Flush the store to disk.
PropertyStore->Flush();
}
}
void FCrashReportAnalyticsSessionSummary::OnApplicationWillTerminate()
{
LogEvent(FString::Printf(TEXT("CRC/Terminate:%s"), *FDateTime::UtcNow().ToString()));
}
void FCrashReportAnalyticsSessionSummary::OnHandleSystemError()
{
CrcAnalyticsProperties::IsCrashing.Set(PropertyStore.Get(), true);
LogEvent(FString::Printf(TEXT("CRC/SysError:%s"), *FDateTime::UtcNow().ToString()));
}
void FCrashReportAnalyticsSessionSummary::OnCrashReportStarted(ECrashContextType CrashType, const TCHAR* ErrorMsg)
{
CrashReportStartTimeSecs = FPlatformTime::Seconds();
// Reset the timers for the current crash report (a safety measure).
CrashReportCollectingStartTimeSecs = CrashReportStartTimeSecs;
CrashReportStackWalkingStartTimeSecs = CrashReportStartTimeSecs;
CrashReportGatheringFilesStartTimeSecs = CrashReportStartTimeSecs;
CrashReportSignalingRemoteAppTimeSecs = CrashReportStartTimeSecs;
CrashReportProcessingStartTimeSecs = CrashReportStartTimeSecs;
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
CrcAnalyticsProperties::IsReportingCrash.Set(PropertyStore.Get(), true);
CrcAnalyticsProperties::ReportCount.Update(PropertyStore.Get(), [](uint32& Actual) { ++Actual; return true; });
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
LogEvent(FString::Printf(TEXT("Report/Start:%s"), *FDateTime::UtcNow().ToString()));
// Log the assert and ensure condition/file/line/message to the diagnostic log gathered by the analytics to enable searching/grouping them later on.
if (CrashType == ECrashContextType::Assert)
{
CrcAnalyticsProperties::AssertCount.Update(PropertyStore.Get(), [](uint32& Actual) { ++Actual; return true; });
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
LogEvent(FString::Printf(TEXT("Assert/Msg: %s"), ErrorMsg));
}
else if (CrashType == ECrashContextType::Ensure)
{
CrcAnalyticsProperties::EnsureCount.Update(PropertyStore.Get(), [](uint32& Actual) { ++Actual; return true; });
// Ensure messages include the ensure call stack. That's not useful for analytics, try keeping the essential only (ensure condition, file, line)
FRegexPattern Pattern(TEXT(R"(.*\[File:.*\]\s*\[Line:\s\d+\])")); // Need help with regex? Try https://regex101.com/
FRegexMatcher Matcher(Pattern, ErrorMsg);
if (Matcher.FindNext())
{
LogEvent(FString::Printf(TEXT("Ensure/Msg: %s"), *Matcher.GetCaptureGroup(0)));
}
else
{
LogEvent(FString::Printf(TEXT("Ensure/Msg: %s"), ErrorMsg));
}
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
}
else if (CrashType == ECrashContextType::Stall)
{
CrcAnalyticsProperties::StallCount.Update(PropertyStore.Get(), [](uint32& Actual) { ++Actual; return true; });
}
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
}
void FCrashReportAnalyticsSessionSummary::OnCrashReportCollecting()
{
CrashReportCollectingStartTimeSecs = FPlatformTime::Seconds();
CrcAnalyticsProperties::IsCollectingCrash.Set(PropertyStore.Get(), true);
FCrashReportAnalyticsSessionSummary::Get().LogEvent(TEXT("Report/Collect"));
}
void FCrashReportAnalyticsSessionSummary::OnCrashReportRemoteStackWalking()
{
CrashReportStackWalkingStartTimeSecs = FPlatformTime::Seconds();
}
void FCrashReportAnalyticsSessionSummary::OnCrashReportGatheringFiles()
{
CrashReportGatheringFilesStartTimeSecs = FPlatformTime::Seconds();
}
void FCrashReportAnalyticsSessionSummary::OnCrashReportSignalingAppToResume()
{
CrashReportSignalingRemoteAppTimeSecs = FPlatformTime::Seconds();
}
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
void FCrashReportAnalyticsSessionSummary::OnCrashReportProcessing(bool bUserInteractive)
{
CrcAnalyticsProperties::IsCollectingCrash.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::IsProcessingCrash.Set(PropertyStore.Get(), true);
CrashReportProcessingStartTimeSecs = FPlatformTime::Seconds();
bProcessingCrashUnattended = !bUserInteractive;
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
FCrashReportAnalyticsSessionSummary::Get().LogEvent(FString::Printf(TEXT("Report/Process:%s"), bUserInteractive ? TEXT("Interactive") : TEXT("Unattended")));
}
void FCrashReportAnalyticsSessionSummary::OnCrashReportCompleted(bool bSubmitted)
{
CrcAnalyticsProperties::IsCollectingCrash.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::IsProcessingCrash.Set(PropertyStore.Get(), false);
CrcAnalyticsProperties::IsReportingCrash.Set(PropertyStore.Get(), false);
double CurrTimeSecs = FPlatformTime::Seconds();
// Total time required to remote stack walk, gather files, respond to the monited app.
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
double CollectTimeSecs = CrashReportProcessingStartTimeSecs - CrashReportCollectingStartTimeSecs;
// Total time required to remote stack walk
double StackWalkSecs = CrashReportGatheringFilesStartTimeSecs - CrashReportStackWalkingStartTimeSecs;
// Total time required to gather files (copy log and generates minidump).
double GatherFileSecs = CrashReportSignalingRemoteAppTimeSecs - CrashReportGatheringFilesStartTimeSecs;
// Total time required by CRC to process the crash report (resolve symbols + showing the UI + user time to respond if the report is interactive).
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
double ProcessTimeSecs = CurrTimeSecs - CrashReportProcessingStartTimeSecs;
// Total time CRC main thread was used to process the crash.
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
double TotalTimeSecs = CurrTimeSecs - CrashReportStartTimeSecs;
if (bProcessingCrashUnattended) // No UI shown to the user that could amplify the time.
{
CrcAnalyticsProperties::LonguestUnattendedReportSecs.Set(PropertyStore.Get(), static_cast<float>(TotalTimeSecs), [](const float* Actual, const float& Proposed) { return Actual == nullptr || Proposed > *Actual; });
}
bProcessingCrashUnattended = false;
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
const TCHAR* Event = bSubmitted ? TEXT("Report/Sent") : TEXT("Report/Discarded");
LogEvent(FString::Printf(TEXT("%s:Walk=%.1fs:Gather=%.1fs:Collect=%.1fs:Process=%.1fs:Total=%.1fs"), Event, StackWalkSecs, GatherFileSecs, CollectTimeSecs, ProcessTimeSecs, TotalTimeSecs));
Generalized the Editor analytics summary session system to be usable/extendable by other apps. Engine/Editor changes: - Split the Editor summary session in two, one summary for the Engine properties and one for the Editor specific properties. Made it easy to extend the Engine summary to create other summaries. - Made the summary sender as agnostics as possible of the keys it sends. - Fixed the system wide lock contention between the process when persisting a session. (On problem caused by the lock is UE-114315). - Fixed concurrent issue when saving the summary sessions on Linux/Mac - Fixed performance issue when saving the summary session on Linux/Mac. This enable saving at higher frequency. - Fixed cases where the same session summary is sent more than once. - Fixed Windows registry key overflow that could happens if we accumulated too many sessions (in theory, this can happen) - Made adding new properties to the summary easy and private to the implementation. - Brought the Linux/Mac implementation closer to Windows implementation. - Reduced memory allocation, especially when the session records a crash. - Improved chances to send the summary non-delayed by allowing the Editor to send the reports if CRC died unexpectedly. - Generalized the support to collect and aggregate analytics from helper processes. For example, CRC already collects analytics that is merged with the Editor summary as information supplement - Reserved the disk space required to store the summary ahead of time to prevent failing later. - Increased frequency at which the summary is persisted because saving the summary is more efficient. (About every 10 seconds rather than every minutes). - Added unit tests CrashReportClient changes: - Created a 'session summary' from the CRC point of view to merge with the Editor summary. - Moved analytics collection in a separated class to make the crash reporting code leaner and less noisy with all the analytics - Merged the CRC diagnostic logger in the class collecting CRC analytics summary and make the diagnostic log a property in the summary. - Collected analytics (on behalf of Editor) in a background thread because CRC main thread can be blocked collecting a crash, so it doesn't pay attention to other things - Added MonitorBatteryLevel and MonitorOnACPower summary properties on Windows. Collected on CRC background thread (never blocked, so we reduce changes to miss the battery running out) - Added MonitorSessionDuration summary property to track now long CRC ran. - Added MonitorQuitSignalRecv summary property to detect when CRC is soft killed like: taskkill /PID 1234 - Added MonitorIsReportingCrash summary property to track when CRC dies reporting a crash. - Added MonitorIsCollectingCrash summary property to track when CRC dies collecting a crash artifacts. - Added IsProcessingCrash summary property to track when CRC dies processing a crash. - Added MonitorCrashed summary property to track when CRC exception handler was triggered. - Added MonitorWasShutdown summary property to track when CRC summary was shutdown - Added MonitorLoggingOut summary property to track when CRC died because the user was logging out (or as result of shutting down or restarting the computer). - More accurate value for DeathTimestamp summary property because this is now captured in CRC background thread (which cannot be busy handling a crash) - Added crash processing timing to CRC diagnostic logs (how long it takes to collect/process a crash). #rb Jamie.Dale, Wes.Hunt, Johan.Berg #jira UETOOL-3500 #jira UE-114315 [CL 16324612 by Patrick Laflamme in ue5-main branch]
2021-05-13 21:58:20 -04:00
}
bool FCrashReportAnalyticsSessionSummary::UpdatePowerStatus()
{
bool bShouldFlush = false;
// Monitor the power level.
TOptional<bool> ConnectedToACPower;
TOptional<uint32> BatteryPercentage;
if (CrashReportClientUtils::GetPowerStatus(ConnectedToACPower, BatteryPercentage))
{
if (ConnectedToACPower)
{
CrcAnalyticsProperties::IsOnACPower.Set(PropertyStore.Get(), *ConnectedToACPower);
}
if (BatteryPercentage)
{
CrcAnalyticsProperties::BatteryLevel.Set(PropertyStore.Get(), *BatteryPercentage, [&bShouldFlush](const uint32* PreviousLevel, const uint32& NewLevel)
{
// Detects when the batter level goes from above to below or equal a threshonld.
constexpr uint32 LowBatteryPct = 2;
if (PreviousLevel && *PreviousLevel > LowBatteryPct && NewLevel <= LowBatteryPct)
{
// Last time, it was above the threshold, but now it dipped below, save the state before the battery runs out.
bShouldFlush = true;
}
return true;
});
}
}
return bShouldFlush;
}