Commit Graph

231 Commits

Author SHA1 Message Date
Andriy Tylychko
31c369c3bf Two related parts:
1) TlsAutoCleanup: in the previous version `FTlsAutoCleanup` used `FRunnableThread` to store registered instances and to delete them on `FRunnableThread` destruction. This means that all threads that are not created by `FRunnableThread` (the game thread in particular, and all 3rd-party threads) never delete registered `FTlsAutoCleanup` instances. This change moves the storage from `FRunnableThread` to TLS to cover all threads

Thsi change can break systems that implemented a workaround for this problem, e.g. by manually deleting registered `FTlsAutoCleanup` instances on the game thread exit, as now this instances will be deleted twice. The solution is to remove the workaround and to rely on uniform `FTlsAutoCleanup` behaviour on all threads.

#rb steve.robb

2) removed CpuProfilerTrace workaround to handle the fact that FTlsAutoCleanup leaks on the game thread, as this is fixed by (1).

#rb martin.ridgers

Need to submit both parts together as they need each other

#robomerge Release-5.0-EarlyAccess

[CL 15624597 by Andriy Tylychko in ue5-main branch]
2021-03-05 11:29:40 -04:00
Johan Berg
cdc73cbb05 [TraceLog] Use 64-bit cycle fields
Back out of previous new base cycle event change and use the absolute 64-bit cycle value instead. The previous solution did not support late connect scenarios. The memory cost is not significant based on the current marker frequency.

#rb martin.ridgers

[CL 15623871 by Johan Berg in ue5-main branch]
2021-03-05 10:06:56 -04:00
Catalin Dragoiu
bd4cde09c9 Change Frame events into a single event with absolute timestamps
#rb Ionut.Matasaru #jira UE-109217

[CL 15622895 by Catalin Dragoiu in ue5-main branch]
2021-03-05 06:02:02 -04:00
Johan Berg
d44f190990 [TraceLog] Fix wrapping memory markers.
#jira UE-107432
#rb martin.ridgers

[CL 15605523 by Johan Berg in ue5-main branch]
2021-03-04 10:00:38 -04:00
Matt Peters
5bcb2860bc #jira UE-109864
Fix compile error in UnrealMultiUserServer - LLM defines need to be wrapped in (), and tagtrace needs to check for ENABLE_LOW_LEVEL_MEM_TRACKER.
#rb Jimmy.Andrews
#rnx

[CL 15586298 by Matt Peters in ue5-main branch]
2021-03-03 15:22:22 -04:00
geoff evans
f8e05cdce4 Don't log non-reporting stalls
#ROBOMERGE-SOURCE: CL 15571654 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)

[CL 15571657 by geoff evans in ue5-main branch]
2021-03-02 14:23:40 -04:00
Matt Peters
215a61a8a5 LowLevelMemTracker: Fix missing declarations. Change FLLMTagDeclaration to allow multiple listeners.
#rb Danny.Couture
#rnx

[CL 15570787 by Matt Peters in ue5-main branch]
2021-03-02 13:15:45 -04:00
Martin Ridgers
b8ed8ba3d4 When capturing and reporting callstacks, use the return address of a failure instead of a count of stack frames to trim. The count approach was spread about in many places and fragile to maintain as code changed. This resulted in "noisy" callstacks with distracting boilerplate present like assert dispatch functions.
#rb brandon.schaefer,will.damon,johan.berg
#rnx

#ushell-cherrypick of 15568119 by Martin.Ridgers

[CL 15568152 by Martin Ridgers in ue5-main branch]
2021-03-02 07:48:13 -04:00
martin ridgers
97f7957704 Log that we're going to stop the trace before stopping the trace so that the stopping trace log is at the end of the trace.
#jira UE-109242
#rnx

#ROBOMERGE-SOURCE: CL 15526946 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)

[CL 15527028 by martin ridgers in ue5-main branch]
2021-02-25 05:41:08 -04:00
Marc Audy
9753392e2b Merge UE5/RES CL# 15462083 to UE5/Main
This represents UE4/Main @ 15414221

[CL 15463811 by Marc Audy in ue5-main branch]
2021-02-18 18:13:28 -04:00
geoff evans
7017731f32 Only initialize stall detector for UnrealEditor.exe (remove it from cooks)
Add function to query if the static API is init and thread should be running
Tidy up some direct checks to the init count to use the new IsRunning function
Make init reference count signed and check for it going negative (mismatched startup/shutdown calls)
Disclaimer: init reference count isn't really used yet, but I'm hoping to propose using it more broadly in the future

#rb francis.hurteau
#jira none
#preflight /job/602c184cada6960001aa3e12

#ROBOMERGE-SOURCE: CL 15425482 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)

[CL 15426390 by geoff evans in ue5-main branch]
2021-02-16 20:04:22 -04:00
geoff evans
f18b2186af Demote StallDetector log prints from Warning to Log.
For now we don't want QA/Automation test results to write JIRAs for these events, we will triage them offline using analytics.

#rb trivial
#jira UE-108077

#ROBOMERGE-SOURCE: CL 15412939 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)

[CL 15412952 by geoff evans in ue5-main branch]
2021-02-15 20:19:36 -04:00
Johan Berg
38389fddd8 [TraceLog] Mark memory initialization event as important
Memory initialization event needs to be marked as important because we cannot receive time marker events before setting the base timestamp. It leads to negative time values.

#rb ionut.matasaru

[CL 15405613 by Johan Berg in ue5-main branch]
2021-02-15 04:40:15 -04:00
geoff evans
98adde3ab8 Support inert FStallDetector objects.
Add support for detectors that are constructed outside of when the StallDetector system is started up.
This gives us the flexibility to startup and shutdown within a finer scope of process startup and shutdown coming in a future change.

#rb francis.hurteau
#jira none
#preflight /job/6025c98c095d790001cca8f5

#ROBOMERGE-SOURCE: CL 15398967 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)

[CL 15398973 by geoff evans in ue5-main branch]
2021-02-12 15:42:45 -04:00
geoff evans
211e371099 Move up StallDetector shutdown into PreExit to shutdown before exceptions in OnExit broadcast.
Use SleepNoStats to prevent stat accumulation of sleep time
Remove CPUTRACE macros to remove dependency on that system (they weren't really useful)
Improve StallDetector logging

#rb francis.hurteau
#jira none
#preflight /job/602195055cb1b300015c6218

#ROBOMERGE-SOURCE: CL 15361323 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)

[CL 15361335 by geoff evans in ue5-main branch]
2021-02-08 18:41:28 -04:00
Johan Berg
16a9acd716 [TraceLog] Move OnExit register later in boot
CallstackTrace was register a callback for OnExit during static init, but this callback could be overwritten due to initialization order. Move the registering later to fix this.

#rnx
#jira UE-107085
#rb paul.chipchase

[CL 15300711 by Johan Berg in ue5-main branch]
2021-02-03 15:18:47 -04:00
Marc Audy
cac1fe0019 Merge UE5/Release-Engine-Staging @ CL# 15299266 to UE5/Main
This represents UE4/Main @ CL# 15277572

[CL 15299962 by Marc Audy in ue5-main branch]
2021-02-03 14:57:28 -04:00
geoff evans
58022cbe85 Add StallDetector to Session Summary for Analytics
This adds some scalar statistics to the session summary:
* The total stall trigger count
* The total reported stalls
* The top stall name
* The top stall budget time in seconds
* The top stall cumulative overage time in seconds
* The top stall trigger count

#jira none
#rb patrick.laflamme, francis.hurteau
#preflight /job/6019a5ef797f69000126f9b6

[CL 15284567 by geoff evans in ue5-main branch]
2021-02-02 16:47:04 -04:00
Johan Berg
1e44893002 [TraceLog] Clean shutdown of callstack tracing worker thread
On process exit the worker thread is killed, but we need orderly shutdown to avoid dropped callstacks and deadlocks when allocating during static deinitialization.

#rb ionut.matasaru, martin.ridgers
#rnx
#jira UE-107085

[CL 15237564 by Johan Berg in ue5-main branch]
2021-01-28 07:15:32 -04:00
geoff evans
4d6be6faf1 Fix clang compile error. Clang's std::atomic copy ctor is deleted
#jira UE-107295
#rb trivial

[CL 15218141 by geoff evans in ue5-main branch]
2021-01-27 12:58:26 -04:00
geoff evans
4f72b503b8 Add Stall Detector API, enabled only for Editor builds for Windows
This code is meant to help locate and send reports/telemetry for slow code pathways that create unresponsive conditions
FGameThreadHitchHeartBeatThreaded was considered, but doesn't fit Editor's needs because its designed around general GameThread deadlines
Editor workloads are much less homogenous, and proper async support for a consistent GameThread deadline in Editor is a ways away
This necessitates a more focused approach where we can instrument specific routines such that each issue their own telemetry report
Add a "Stalls" counter in the Frame Rate and Memory title bar stats
Add LogStall Log category for viewing details about stalls that have occurred
Introduces a stall counter object on the GameThread to collect statistical data about stalls (this will not report to telemetry)
Future changes will introduce report objects into specific routines to upload to crashreporter
Future changes will introduce support for non-Windows OSes

#jira none
#rb francis.hurteau

[CL 15213394 by geoff evans in ue5-main branch]
2021-01-26 20:26:53 -04:00
Johan Berg
f16616d73b Fix for nonunity builds
#rb none
#rnx

[CL 15201567 by Johan Berg in ue5-main branch]
2021-01-26 04:48:20 -04:00
Johan Berg
7dfe9bb1c1 [Trace] Removed use of TMiniArray.
#rb martin.ridgers
#rnx

[CL 15162897 by Johan Berg in ue5-main branch]
2021-01-22 05:44:46 -04:00
Martin Ridgers
abc747fc2e Inform the user if trace.start was not possible. There exist some plugins that create their own trace store and sequester the runtime trace. As tracing is process wide this ties up the trace system such that other use-cases like trace.start can't be satisfied. When this scenario is encountered will inform the user and invite them to run trace.stop first.
#jira ue-104408
#rb jb

[CL 15162718 by Martin Ridgers in ue5-main branch]
2021-01-22 04:21:00 -04:00
Martin Ridgers
ae038f291d It is okay to call GAux.EnableChannels() directly here.
[CL 15153557 by Martin Ridgers in ue5-main branch]
2021-01-21 08:31:32 -04:00