### Features
We can now write tailing memory to a utrace file. Snapshotting while tracing to a file or socket is supported.
This change introduces a new console command: Trace.SnapshotFile. It writes the profiling data from memory into an optionally specified a utrace file.
### Testing
Run Editor with -tracefile
Run Editor with -trace, connect live with insights
Run Editor without -trace (this does generate a very small utrace since there is memory and data in the tail even though default channels are not enabled)
#rb johan.berg
#jira UE-150302,UE-150292
#preflight 6272a3213e1f2a9d3a88ee3d
#ROBOMERGE-OWNER: geoff.evans
#ROBOMERGE-AUTHOR: geoff.evans
#ROBOMERGE-SOURCE: CL 20124258 via CL 20125717 via CL 20126082
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)
[CL 20129506 by geoff evans in ue5-main branch]
This should fix the case where performance counters are set but none of them do Begin / End CSV calls or potentially do multiple calls.
Also add logging to FCsvProfiler::EndFrame() so we can tell when ECsvCommandType::Stop is processed.
#rnx
#rb Ilya.Loshchinin
[FYI] Ben.Woodhouse, Bart.Hawthorne, Andrew.Ladenberger
#preflight 626eb998c731e279c32db6ba
#ROBOMERGE-AUTHOR: michael.sartain
#ROBOMERGE-SOURCE: CL 20041664 via CL 20042062 via CL 20042078 via CL 20042092 via CL 20042112
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)
[CL 20043206 by michael sartain in ue5-main branch]
### Features
This change enables the StallDetector watchdog in Editor to submit reports to crashreporter about threads violating instrumented deadlines in the source code. This feature was available prior on Windows, and this change adds Linux support.
### Notes
New APIs:
ReportStall()
CaptureThreadPortableCallStack()
Many APIs are updated from purely "Ensure" naming to more general naming. Stalls are more like Ensures than crashes, and so the appropriate renames to make the code readable and clear have been made. In some places Ensure is replaced with the clearer: Continuable Event nomenclature.
### Testing
I synthesized an ensure on Linux, and did the same for a stall. I then compared crash report XML file to make sure they contain accurate data in the callstack, portable callstack, and other fields in the report. I also noted that the stall information was showing as expected in the crash reporter.
#rb brandon.schaefer, francis.hurteau
#jira UETOOL-3336
#preflight 625e20d2804460ab0fea3277
[CL 19911608 by geoff evans in ue5-main branch]
First one because due to RTC in EOS not having frame pointers causing a crash when walking down the stack. Limiting callstack tracing with a scope.
The second one because of limited stack size (4k) in a Curl Helper thread, causing a crash when walking down the stack because the callstack walker was allocating 4k on the stack. Doing it inplace to allocate only 2k.
[REVIEW] [at]johan.berg
[FYI] pavel.punsky
#rnx
#ROBOMERGE-OWNER: john.huelin
#ROBOMERGE-AUTHOR: john.huelin
#ROBOMERGE-SOURCE: CL 19739450 via CL 19739526 via CL 19739543 via CL 19739742 via CL 19739765
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)
[CL 19741582 by john huelin in ue5-main branch]
Manual memory tracking in LLM has a feature for defining a "fallback tag" for the allocation if no tag is currently set. This change adds a new scope type which allows us to express this behaviour in memory tracing as well.
#jira UE-142692
#rb ionut.matasaru
#preflight 624ed4e9bd5b36eec338a937
[CL 19667413 by Johan Berg in ue5-main branch]
#rb johan.berg
#jira UE-145980
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19387487 via CL 19388738 via CL 19390713 via CL 19398886 via CL 19399041
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)
[CL 19403777 by robert millar in ue5-main branch]
- Added a range allocator for allocating small blocks. It is enabled if 2 MB GPU page is used to reduce memory overhead. Perf is on par with FGPUMemoryPage.
- 20-run ReplayRun test data with 2 MB page and the new allocator: -0.12 ms FrameTime Avg, -0.17 ms GameThreadTime Avg, -0.43 ms RenderThreadTime Avg, -0.12 ms RHIThreadTime Avg, +0.6 DynRes % Avg, and +18 MemFreeMB Min.
- Added support for recording and replaying FMemBlock allocations. Courtesy to Ben for providing most of the code.
- Small adjustment to how MemBlockStat_Wasted is calculated. It now takes unused page space into account in addition to alignment paddings.
- 2 MB GPU page is enabled by default but can be disabled through a project define.
#rb luke.thatcher, ben.woodhouse
#ROBOMERGE-AUTHOR: jian.ru
#ROBOMERGE-SOURCE: CL 19065260 via CL 19065261 via CL 19065262 via CL 19083385 via CL 19084250 via CL 19086050
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)
[CL 19124776 by jian ru in ue5-main branch]
Adds a define UE_FNAME_OUTLINE_NUMBER.
Removes of FName, FMinimalName from memory image support. Adds of FMemoryImageName.
Removal of FMinimalName operator<<, all fields made private, size made variable.
All fields of FScriptName made private.
Added console commands for dumping numbered/unnumbered names and stats.
#rb johan.torp
#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19058026 via CL 19058611 via CL 19058656 via CL 19061727 via CL 19061740 via CL 19064047
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)
[CL 19066701 by robert millar in ue5-main branch]
Clear caches and update a canary value across the fork boundary to avoid touching pages that were allocated pre-fork to minimize unnecessary duplication of shared pages.
Savings depend on pre-fork fragmentation/slack and number of concurrent child processes.
#rb louisphilippe.seguin, johan.torp, arciel.rekman
#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19057122 via CL 19057644 via CL 19057693 via CL 19057718 via CL 19057737 via CL 19059617
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)
[CL 19066314 by robert millar in ue5-main branch]
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 18847479 via CL 18848589 via CL 18848624 via CL 18854020 via CL 18854481
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v910-18824042)
[CL 18854518 by ben woodhouse in ue5-main branch]
- this change prevents gauntlet from running tasks on Xbox so I am backing it out
- instance of Horde failure: https://horde.devtools.epicgames.com/job/61f43b2f74510448a686da51?step=72cc
- the XboxConsoleInputReader thread is created and immediately terminates
- this change introduced a null out of Runnable in the Xbox version of FRunnableThreadWin::Run()
- this causes FRunnableThread::SetupCreatedThread() to assert on startup that a valid runnable thread was created but has no Runnable
#rb none
#preflight 61f47fb6801201ab389bb311
[FYI] ionut.matasaru
Original CL Desc
-----------------------------------------------------------------
[Insights]
- Fixed memory tracking for an FRunnableThread to emit a free event when the thread ends.
- Fixed memory tracking for "ProgramSize" of loaded modules (dlls), to emit a free event when dll is unloaded.
#jira UE-140358
#rb Johan.Berg
#preflight 61f40923f628752a568b292b
#ROBOMERGE-AUTHOR: eric.mcdaniel
#ROBOMERGE-SOURCE: CL 18781656 in //UE5/Release-5.0/... via CL 18781895 via CL 18781943
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)
[CL 18781953 by eric mcdaniel in ue5-main branch]
- Fixed parsing of Trace.Start/Trace.File console command params to correctly detect "file path" and "list of channels" parameters.
- Fixed Trace.Disable to not save anymore the previous enabled channels as "paused" preset.
- Fixed Trace::Connect (Tarce.Send, Trace.File) to not resume paused channels anymore.
#jira UE-140495
#rb none
#preflight 61f448837a7f0d39ddf2d923
#ROBOMERGE-AUTHOR: ionut.matasaru
#ROBOMERGE-SOURCE: CL 18777275 in //UE5/Release-5.0/... via CL 18778397 via CL 18778925
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)
[CL 18778961 by ionut matasaru in ue5-main branch]
- Fixed memory tracking for an FRunnableThread to emit a free event when the thread ends.
- Fixed memory tracking for "ProgramSize" of loaded modules (dlls), to emit a free event when dll is unloaded.
#jira UE-140358
#rb Johan.Berg
#preflight 61f40923f628752a568b292b
#ROBOMERGE-AUTHOR: ionut.matasaru
#ROBOMERGE-SOURCE: CL 18771801 in //UE5/Release-5.0/... via CL 18771806 via CL 18771881
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)
[CL 18771891 by ionut matasaru in ue5-main branch]
#preflight 61f40bd6241aeec8655410c0
#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18771777 in //UE5/Release-5.0/... via CL 18771791 via CL 18771877
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)
[CL 18771890 by johan berg in ue5-main branch]
Using auto connect would always enable default channel, even if started with a specific channel. Also renamed internal type to avoid confusion with TraceLog's FChannel.
#rb ionut.matasaru
#preflight 61f40a3074510448a6818204
#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18771519 in //UE5/Release-5.0/... via CL 18771526 via CL 18771686
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)
[CL 18771706 by johan berg in ue5-main branch]
