Commit Graph

150 Commits

Author SHA1 Message Date
robert millar
30ed588107 Back out CsvProfiler changes: Add SetAndHold command to csv profiler custom stats to allow infrequently updated statistics to have their value persisted.
[FYI] ben.woodhouse

#ROBOMERGE-SOURCE: CL 16554462 via CL 16554473 via CL 16554480 via CL 16554514
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v828-16531559)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 16554515 by robert millar in ue5-release-engine-test branch]
2021-06-03 18:19:13 -04:00
robert millar
3a4438b84d Add SetAndHold command to csv profiler custom stats to allow infrequently updated statistics to have their value persisted.
#ROBOMERGE-SOURCE: CL 16509803 via CL 16509813 via CL 16509817 via CL 16509820
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v826-16501804)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 16509821 by robert millar in ue5-release-engine-test branch]
2021-05-31 12:34:52 -04:00
Marc Audy
e80ea6b959 Merge from Release-Engine-Staging @ 16444985
This represents UE4/Main @ 16445039 and Dev-PerfTest @ 16444526

[CL 16445122 by Marc Audy in ue5-release-engine-test branch]
2021-05-25 02:43:26 -04:00
Michael Noland
3902270d03 Fixed a type truncation warning in the stall detector (the warning was temporarily globally disabled in UE5 main, which got fixed today)
#codereview geoff.evans

[CL 16421755 by Michael Noland in ue5-main branch]
2021-05-21 14:17:07 -04:00
martins mozeiko
2208dade76 Add context switch and stack sampling trace collection on Windows using ETW
#rb Ionut.Matasaru

[CL 16357198 by martins mozeiko in ue5-main branch]
2021-05-17 15:28:50 -04:00
geoff evans
e1d1984bd4 Add support for trace counters to stall detector and trace summarize commandlet for telemetry in Iteration Profile
StallDetector:
* Replace static and per-callsite counter variables with trace classes
* Compose a trace api appropriate counter name (Stall Detector category)

SummarizeTraceCommandlet:
* Add support for extract and decode of counter messages from utrace files with a new analyzer class
* Report counters with name and count columns in the existing csv
* Support generation of counter events and values through to telemetry csv

#jira UETOOL-3506
#rb francis.hurteau
#preflight 609c5f45e88e9100010b5466

[CL 16314462 by geoff evans in ue5-main branch]
2021-05-13 12:37:02 -04:00
ionut matasaru
4690d6b6ca Fixed missing REOPEN macros when PLATFORMFILETRACE_ENABLED is 0.
#rb Catalin.Dragoiu

[CL 16310865 by ionut matasaru in ue5-main branch]
2021-05-13 04:40:24 -04:00
ionut matasaru
2d5c799e12 [Insights]
- Added new trace events to track the ReOpenFile API. Updated the PlatformFileTraceAnalysis to process the respective new trace events. This fixes the issue where activities were displayed in Unreal Insights as I/O events with "Unknown" file (UE-114506).
  - Updated warning messages and debug code in PlatformFileTraceAnalysis.
  - Added handle tracking in Insights for I/O Activity events. File handle and read/write handles are now displayed in tooltip of an I/O Activity event.
  - Added "ReOpen" as a separate I/O activity type.

#jira UE-114506
#rb Catalin.Dragoiu, Robert.Millar

[CL 16310259 by ionut matasaru in ue5-main branch]
2021-05-13 03:13:34 -04:00
Andrew Davidson
3ddc3a4da3 Merge up from //UE5/Dev-LargeWorldCoordinates
#rb none

[CL 16211417 by Andrew Davidson in ue5-main branch]
2021-05-05 15:07:25 -04:00
aurel cordonnier
50944fd712 Merge UE5/RES @ 16162155 to UE5/Main
This represents UE4/Main @ 16130047 and Dev-PerfTest @ 16126156

[CL 16163576 by aurel cordonnier in ue5-main branch]
2021-04-29 19:32:06 -04:00
Richard Malo
3930c3f107 Improved WorldPartition debugging tools:
- Added wp.Runtime.ToggleDrawRuntimeCellsDetails : Lists all world partition runtime streaming cells with their name and grouped by streaming state
- Added world partition debug filters (by datalayer(s), streaming status, grid name(s), cell name(s)) that affects debug display (runtime hash 2d/3d, cell details)
- Added Data Layer streaming status in 2d/3d runtime hash debug draw
- Added Legends with colors and minimal stats for DataLayers and Runtime Streaming Status
#rb jeanfrancois.dube, patrick.enfedaque

[CL 16157856 by Richard Malo in ue5-main branch]
2021-04-29 11:43:08 -04:00
ben woodhouse
47a30fd5de Mac/Linux build fix
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16110729 via CL 16110730 via CL 16110731 via CL 16110732 via CL 16110733 via CL 16110734
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)

[CL 16117118 by ben woodhouse in ue5-main branch]
2021-04-26 14:43:02 -04:00
ben woodhouse
5f00ce5cd9 CsvProfiler metadata improvements
- Implement ability to unset metadata values by setting with nullptr
- Sanitize metadata to remove commas from values and keys (the commandline is excluded because it's special)
- Set a starttimestamp metadata value (in seconds since the Unix epoch).
- Add metadata for OS, CPU, Namedevents (at capture start)
- Log metadata changes
[FYI] james.doverspike,andrew.ladenberger

#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16110194 via CL 16110198 via CL 16110202 via CL 16110209 via CL 16110215 via CL 16110345
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)

[CL 16117042 by ben woodhouse in ue5-main branch]
2021-04-26 14:40:19 -04:00
danny couture
6446003110 Add Concurrency Viewer as an external profiler and compile only when extension is installed in VS
Avoid activating any external profiler by default as this could cause slowdown depending on which profiler gets randomly activated

#rb Martin.Ridgers

[CL 15861416 by danny couture in ue5-main branch]
2021-03-30 09:36:28 -04:00
johan berg
7eed4b9b0e [TraceLog] Fix channel presets early initialization
In a previous change TraceAux::Initialize was moved to a point very early in process lifetime. At this point the config system is not ready, so resolving data driven channel presets does not work. To solve this we make two changes, define two presets in code (default and memory) which require very early application, and add a second pass of channel initialization which is executed directly after config system is up.

#rb martin.ridgers

#ROBOMERGE-OWNER: johan.berg
#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 15822904 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v785-15821902)
#ROBOMERGE-CONFLICT from-shelf

[CL 15829754 by johan berg in ue5-main branch]
2021-03-25 15:39:34 -04:00
Marc Audy
0cbbc781ca Merge UE5/Release-Engine-Staging @ 15740152 to UE5/Main
This represents UE4/Main @ 15709114

[CL 15740605 by Marc Audy in ue5-main branch]
2021-03-18 15:20:03 -04:00
Andriy Tylychko
31c369c3bf Two related parts:
1) TlsAutoCleanup: in the previous version `FTlsAutoCleanup` used `FRunnableThread` to store registered instances and to delete them on `FRunnableThread` destruction. This means that all threads that are not created by `FRunnableThread` (the game thread in particular, and all 3rd-party threads) never delete registered `FTlsAutoCleanup` instances. This change moves the storage from `FRunnableThread` to TLS to cover all threads

Thsi change can break systems that implemented a workaround for this problem, e.g. by manually deleting registered `FTlsAutoCleanup` instances on the game thread exit, as now this instances will be deleted twice. The solution is to remove the workaround and to rely on uniform `FTlsAutoCleanup` behaviour on all threads.

#rb steve.robb

2) removed CpuProfilerTrace workaround to handle the fact that FTlsAutoCleanup leaks on the game thread, as this is fixed by (1).

#rb martin.ridgers

Need to submit both parts together as they need each other

#robomerge Release-5.0-EarlyAccess

[CL 15624597 by Andriy Tylychko in ue5-main branch]
2021-03-05 11:29:40 -04:00
Marc Audy
9753392e2b Merge UE5/RES CL# 15462083 to UE5/Main
This represents UE4/Main @ 15414221

[CL 15463811 by Marc Audy in ue5-main branch]
2021-02-18 18:13:28 -04:00
geoff evans
7017731f32 Only initialize stall detector for UnrealEditor.exe (remove it from cooks)
Add function to query if the static API is init and thread should be running
Tidy up some direct checks to the init count to use the new IsRunning function
Make init reference count signed and check for it going negative (mismatched startup/shutdown calls)
Disclaimer: init reference count isn't really used yet, but I'm hoping to propose using it more broadly in the future

#rb francis.hurteau
#jira none
#preflight /job/602c184cada6960001aa3e12

#ROBOMERGE-SOURCE: CL 15425482 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)

[CL 15426390 by geoff evans in ue5-main branch]
2021-02-16 20:04:22 -04:00
geoff evans
98adde3ab8 Support inert FStallDetector objects.
Add support for detectors that are constructed outside of when the StallDetector system is started up.
This gives us the flexibility to startup and shutdown within a finer scope of process startup and shutdown coming in a future change.

#rb francis.hurteau
#jira none
#preflight /job/6025c98c095d790001cca8f5

#ROBOMERGE-SOURCE: CL 15398967 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)

[CL 15398973 by geoff evans in ue5-main branch]
2021-02-12 15:42:45 -04:00
Marc Audy
cac1fe0019 Merge UE5/Release-Engine-Staging @ CL# 15299266 to UE5/Main
This represents UE4/Main @ CL# 15277572

[CL 15299962 by Marc Audy in ue5-main branch]
2021-02-03 14:57:28 -04:00
geoff evans
7fc8c2c47f Fix for disliked comment style in Clang
#rb trivial
#jira UE-107715

[CL 15298638 by geoff evans in ue5-main branch]
2021-02-03 14:17:11 -04:00
geoff evans
58022cbe85 Add StallDetector to Session Summary for Analytics
This adds some scalar statistics to the session summary:
* The total stall trigger count
* The total reported stalls
* The top stall name
* The top stall budget time in seconds
* The top stall cumulative overage time in seconds
* The top stall trigger count

#jira none
#rb patrick.laflamme, francis.hurteau
#preflight /job/6019a5ef797f69000126f9b6

[CL 15284567 by geoff evans in ue5-main branch]
2021-02-02 16:47:04 -04:00
geoff evans
4f72b503b8 Add Stall Detector API, enabled only for Editor builds for Windows
This code is meant to help locate and send reports/telemetry for slow code pathways that create unresponsive conditions
FGameThreadHitchHeartBeatThreaded was considered, but doesn't fit Editor's needs because its designed around general GameThread deadlines
Editor workloads are much less homogenous, and proper async support for a consistent GameThread deadline in Editor is a ways away
This necessitates a more focused approach where we can instrument specific routines such that each issue their own telemetry report
Add a "Stalls" counter in the Frame Rate and Memory title bar stats
Add LogStall Log category for viewing details about stalls that have occurred
Introduces a stall counter object on the GameThread to collect statistical data about stalls (this will not report to telemetry)
Future changes will introduce report objects into specific routines to upload to crashreporter
Future changes will introduce support for non-Windows OSes

#jira none
#rb francis.hurteau

[CL 15213394 by geoff evans in ue5-main branch]
2021-01-26 20:26:53 -04:00
ionut matasaru
77c26f0fad Fixed comment for TRACE_CPUPROFILER_EVENT_SCOPE macro.
#rb none

[CL 15128209 by ionut matasaru in ue5-main branch]
2021-01-18 07:46:58 -04:00