Commit Graph

378 Commits

Author SHA1 Message Date
jian ru
da3568ff5c PS4 2 MB GPU page tunings:
- Added a range allocator for allocating small blocks. It is enabled if 2 MB GPU page is used to reduce memory overhead. Perf is on par with FGPUMemoryPage.
- 20-run ReplayRun test data with 2 MB page and the new allocator: -0.12 ms FrameTime Avg, -0.17 ms GameThreadTime Avg, -0.43 ms RenderThreadTime Avg, -0.12 ms RHIThreadTime Avg, +0.6 DynRes % Avg, and +18 MemFreeMB Min.
- Added support for recording and replaying FMemBlock allocations. Courtesy to Ben for providing most of the code.
- Small adjustment to how MemBlockStat_Wasted is calculated. It now takes unused page space into account in addition to alignment paddings.
- 2 MB GPU page is enabled by default but can be disabled through a project define.

#rb luke.thatcher, ben.woodhouse

#ROBOMERGE-AUTHOR: jian.ru
#ROBOMERGE-SOURCE: CL 19065260 via CL 19065261 via CL 19065262 via CL 19083385 via CL 19084250 via CL 19086050
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)

[CL 19124776 by jian ru in ue5-main branch]
2022-02-24 15:53:12 -05:00
robert millar
5811293e2a Add TargetRules.bFNameOutlineNumber as an option to reduce the size of FName by 4 bytes by storing the number in the name table.
Adds a define UE_FNAME_OUTLINE_NUMBER.
Removes of FName, FMinimalName from memory image support. Adds of FMemoryImageName.
Removal of FMinimalName operator<<, all fields made private, size made variable.
All fields of FScriptName made private.
Added console commands for dumping numbered/unnumbered names and stats.

#rb johan.torp

#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19058026 via CL 19058611 via CL 19058656 via CL 19061727 via CL 19061740 via CL 19064047
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)

[CL 19066701 by robert millar in ue5-main branch]
2022-02-21 02:35:48 -05:00
robert millar
b4bfddef0a MallocBinned2 fork support.
Clear caches and update a canary value across the fork boundary to avoid touching pages that were allocated pre-fork to minimize unnecessary duplication of shared pages.
Savings depend on pre-fork fragmentation/slack and number of concurrent child processes.

#rb louisphilippe.seguin, johan.torp, arciel.rekman

#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19057122 via CL 19057644 via CL 19057693 via CL 19057718 via CL 19057737 via CL 19059617
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)

[CL 19066314 by robert millar in ue5-main branch]
2022-02-21 02:04:42 -05:00
ilya loshchinin
7bfdf97694 Fixed CSV profiler thread allocating temporary memory and potentially blocking the memory allocator.
#rb louisphilippe.seguin
#preflight 620e66a5ce15efbba68afae0

#ROBOMERGE-AUTHOR: ilya.loshchinin
#ROBOMERGE-SOURCE: CL 19036025 via CL 19036072 via CL 19036095 via CL 19036473
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)

[CL 19036564 by ilya loshchinin in ue5-main branch]
2022-02-17 12:21:25 -05:00
ionut matasaru
06d9ea5567 [Insights] Added hooks for VirtualAlloc2 API for memory tracing.
#jira UE-131214
#rb Johan.Berg
#preflight 61fd0455b5092d45ad1325a4

[CL 18861473 by ionut matasaru in ue5-main branch]
2022-02-04 06:01:58 -05:00
ben woodhouse
c7c995264b Add named events for EventWait stats instrumented via BeginSetWaitStat/EndSetWaitStat if -csvNamedEvents is enabled.
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 18847479 via CL 18848589 via CL 18848624 via CL 18854020 via CL 18854481
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v910-18824042)

[CL 18854518 by ben woodhouse in ue5-main branch]
2022-02-03 18:04:27 -05:00
catalin dragoiu
a2133173d2 Fix non unity build error for Unreal Insights
#rb trivial
#preflight 61f9117ac54b18c42dcef6eb

#ROBOMERGE-AUTHOR: catalin.dragoiu
#ROBOMERGE-SOURCE: CL 18806716 in //UE5/Release-5.0/... via CL 18808598 via CL 18822196
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18822500 by catalin dragoiu in ue5-main branch]
2022-02-02 02:46:56 -05:00
ionut matasaru
a0903e7039 [Insights] Fixed memory tracking for "ProgramSize" of loaded modules (dlls), to emit a free event when dll is unloaded.
#jira UE-140358
#rb Johan.Berg
#preflight 61f796986a7c7b134f2f7aec

#ROBOMERGE-AUTHOR: ionut.matasaru
#ROBOMERGE-SOURCE: CL 18787309 in //UE5/Release-5.0/... via CL 18787313 via CL 18787334
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18787335 by ionut matasaru in ue5-main branch]
2022-01-31 03:11:27 -05:00
eric mcdaniel
361f6b69d1 [Backout] - CL18771801
- this change prevents gauntlet from running tasks on Xbox so I am backing it out
  - instance of Horde failure:  https://horde.devtools.epicgames.com/job/61f43b2f74510448a686da51?step=72cc

- the XboxConsoleInputReader thread is created and immediately terminates
  - this change introduced a null out of Runnable in the Xbox version of FRunnableThreadWin::Run()
    - this causes FRunnableThread::SetupCreatedThread() to assert on startup that a valid runnable thread was created but has no Runnable

#rb none
#preflight 61f47fb6801201ab389bb311
[FYI] ionut.matasaru

Original CL Desc
-----------------------------------------------------------------
[Insights]
  - Fixed memory tracking for an FRunnableThread to emit a free event when the thread ends.
  - Fixed memory tracking for "ProgramSize" of loaded modules (dlls), to emit a free event when dll is unloaded.

#jira UE-140358
#rb Johan.Berg
#preflight 61f40923f628752a568b292b

#ROBOMERGE-AUTHOR: eric.mcdaniel
#ROBOMERGE-SOURCE: CL 18781656 in //UE5/Release-5.0/... via CL 18781895 via CL 18781943
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18781953 by eric mcdaniel in ue5-main branch]
2022-01-28 20:10:09 -05:00
ionut matasaru
c56cc42e9f [Insights]
- Fixed parsing of Trace.Start/Trace.File console command params to correctly detect "file path" and "list of channels" parameters.
  - Fixed Trace.Disable to not save anymore the previous enabled channels as "paused" preset.
  - Fixed Trace::Connect (Tarce.Send, Trace.File) to not resume paused channels anymore.

#jira UE-140495
#rb none
#preflight 61f448837a7f0d39ddf2d923

#ROBOMERGE-AUTHOR: ionut.matasaru
#ROBOMERGE-SOURCE: CL 18777275 in //UE5/Release-5.0/... via CL 18778397 via CL 18778925
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18778961 by ionut matasaru in ue5-main branch]
2022-01-28 16:45:39 -05:00
ionut matasaru
2df95139ab [Insights]
- Fixed memory tracking for an FRunnableThread to emit a free event when the thread ends.
  - Fixed memory tracking for "ProgramSize" of loaded modules (dlls), to emit a free event when dll is unloaded.

#jira UE-140358
#rb Johan.Berg
#preflight 61f40923f628752a568b292b

#ROBOMERGE-AUTHOR: ionut.matasaru
#ROBOMERGE-SOURCE: CL 18771801 in //UE5/Release-5.0/... via CL 18771806 via CL 18771881
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18771891 by ionut matasaru in ue5-main branch]
2022-01-28 10:54:49 -05:00
johan berg
38795db092 [Insights] Fixes for mismatching alloc/free events when using MiMalloc
#preflight 61f40bd6241aeec8655410c0

#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18771777 in //UE5/Release-5.0/... via CL 18771791 via CL 18771877
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18771890 by johan berg in ue5-main branch]
2022-01-28 10:54:42 -05:00
johan berg
9b47e4306b [Insights] Auto connect would always enable default channels.
Using auto connect would always enable default channel, even if started with a specific channel. Also renamed internal type to avoid confusion with TraceLog's FChannel.

#rb ionut.matasaru
#preflight 61f40a3074510448a6818204

#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18771519 in //UE5/Release-5.0/... via CL 18771526 via CL 18771686
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18771706 by johan berg in ue5-main branch]
2022-01-28 10:35:54 -05:00
ionut matasaru
72eff079cd [Insights] Fixed use of UE_MEMSCOPE(TRACE_TAG); to include both the ReallocFree and the ReallocAlloc (events that implies push/pop of tags, so cannot be split by other scoped mem tags).
#rb Johan.Berg
#preflight skip

#ROBOMERGE-AUTHOR: ionut.matasaru
#ROBOMERGE-SOURCE: CL 18736173 in //UE5/Release-5.0/... via CL 18736214 via CL 18736369
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18736407 by ionut matasaru in ue5-main branch]
2022-01-26 11:25:58 -05:00
per larsson
159873fbfb Trace - fix issue tracing to file in root drive directory
#rb Martin.Ridgers
#preflight none

#ROBOMERGE-AUTHOR: per.larsson
#ROBOMERGE-SOURCE: CL 18733390 in //UE5/Release-5.0/... via CL 18733393 via CL 18733428
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18733432 by per larsson in ue5-main branch]
2022-01-26 04:06:43 -05:00
evgenii babinets
8704998a31 Added a few implementations of the ANSI overload of FExternalProfiler::StartScopedEvent. These both support ANSI natively and so we don't need to go through an expensive TCHAR conversion if that is what the build is configured to. This should make it viable to use the ANSI external profiler on all platforms.
#rb jordan.cristiano

#ROBOMERGE-AUTHOR: evgenii.babinets
#ROBOMERGE-SOURCE: CL 18717386 via CL 18717574 via CL 18717635 via CL 18718411 via CL 18718440
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18718451 by evgenii babinets in ue5-main branch]
2022-01-24 20:26:37 -05:00
johan berg
4e9859fe11 [Insights] Clear thread buffer for Cpu scopes
In order to avoid scopes ending up after thread end (rare) clear the thread buffer pointer. In that case a new thread buffer will created, specifically for that scope.

#rb ionut.matasaru
#preflight 61eaaff35f04f4cdf1bae9bd

#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18688576 in //UE5/Release-5.0/... via CL 18688580 via CL 18688590
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18688595 by johan berg in ue5-main branch]
2022-01-21 08:32:31 -05:00
ionut matasaru
a6866d78e1 [Insights]
- Updated help for trace console commands.
  - Fixed comments and code style issues in TraceAuxiliary.

#rb Johan.Berg
#preflight 61ea7ced445cebac10bcd2da

#ROBOMERGE-OWNER: ionut.matasaru
#ROBOMERGE-AUTHOR: ionut.matasaru
#ROBOMERGE-SOURCE: CL 18687495 in //UE5/Release-5.0/... via CL 18687855 via CL 18687859
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18687864 by ionut matasaru in ue5-main branch]
2022-01-21 06:40:36 -05:00
johan berg
a329d56129 [Insights] Always start Trace worker thread, even if SupportsMultithreading is false.
Trace normally runs with a worker thread to write data regularly, however when `FPlatformProcess:SupportsMultithreading` was false we did not start the worker thread, instead we relied on EndFrame delegate to manually pump Trace. When tracing is very intense (e.g. memory tracing) could lead to data building up internally, up to 1 GB of data by the time the first frame callback came. We have recently added a special sync type packet in the stream to deal with edge cases related to late-connect, so analysis stalled until those came trough. However since the backlog of data has to be transmitted first, those sync packet ended up after the initial data. Analysis had to (very inefficiently) get trough that 1Gb of data before the first sync packet came through and events starts to appear.

With this change trace always start the worker thread (if we're intending to fork) regardless of the SupportsMultithreading setting.

#rb ionut.matasaru, ben.woodhouse

#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18675264 via CL 18675395 via CL 18675411 via CL 18675736 via CL 18675867 via CL 18675943
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v902-18672795)

[CL 18679793 by johan berg in ue5-main branch]
2022-01-20 15:27:07 -05:00
johan berg
50b2ee38ab [Insights] When tracing to file all relative paths should go to Profliling dir
When starting a trace to a file, all relative paths should go to the Profiling directory, on Windows even paths that starts with '/'. Also cleaned up includes.

#rb ionut.matasaru
#jira UE-138824
#preflight 61e963e7817064a62d47ddc1

#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18673744 in //UE5/Release-5.0/... via CL 18673759 via CL 18673763
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v902-18672795)

[CL 18673770 by johan berg in ue5-main branch]
2022-01-20 08:45:52 -05:00
johan berg
a56558e662 [Insights] Fix copy paste and build error
* An extra '&' token was removed on the type alias in the no logging case.
 * In the case where trace is enabled, but logging is disabled, made a identical solution in TraceAuxImpl.

#rb ionut.matasaru
#jira UE-139439, UE-139433, UE-139444
#preflight 61e80742843acf1b2429961b

#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18657057 in //UE5/Release-5.0/... via CL 18657068 via CL 18657070
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v900-18638592)

[CL 18657072 by johan berg in ue5-main branch]
2022-01-19 07:58:49 -05:00
johan berg
8faf5c82d8 [Insights] Fix for no logging situations
#rb ionut.matasaru
#preflight 61e586b0873f2ea48f2fdd62

#ROBOMERGE-OWNER: johan.berg
#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18633108 in //UE5/Release-5.0/... via CL 18633166 via CL 18633182
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18637368 by johan berg in ue5-main branch]
2022-01-18 04:30:03 -05:00
johan berg
61dc9c192e [Insights] Allow log output in TraceAux to be routed to arbitrary category.
In order to be able to output detailed error and status messages to both core log or console feedback log depending on situation, add argument to set the target category. Also removed now redundant logs in console commands, since more detailed information is now emitted on the console feedback.
#preflight 61e56f34873f2ea48f23d007

#ROBOMERGE-OWNER: johan.berg
#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18632544 in //UE5/Release-5.0/... via CL 18633016 via CL 18633031
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)
#ROBOMERGE-CONFLICT from-shelf

[CL 18637343 by johan berg in ue5-main branch]
2022-01-18 04:24:47 -05:00
johan berg
bfd98e9be4 [Insights] Append default file name for paths that end in directories.
Append the default file name when the trace target path ends with a slash, for examplethe path "foo/bar/". The target trace file will then be "foo/bar/defaultname.utrace". Note that "foo/bar" will still target "foo/bar.utrace".

#jira UE-138824
#rb ionut.matasaru
#preflight 61e53ecd0c34e9a5ae5f838e

#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18631469 in //UE5/Release-5.0/... via CL 18631663 via CL 18631684
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18636673 by johan berg in ue5-main branch]
2022-01-17 23:59:28 -05:00
johan berg
0f8c7422fa [Insights] Reset commandline channels at every start.
* Whenever starting a trace command (from the api or console command), reset the current command line list of channels.
* Don't assume default channels should be enabled if none are given. For command line arguments this will still be the case, but not console commands or the API.
* Emit log message when auto connecting.
#preflight 61e189669010dadd257b0e38

#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18617306 in //UE5/Release-5.0/... via CL 18617329 via CL 18617340
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18617352 by johan berg in ue5-main branch]
2022-01-14 09:46:50 -05:00