Commit Graph

308 Commits

Author SHA1 Message Date
Martin Ridgers
d40cccbbfe An implementation of LaunchUnrealTrace() for POSIX platforms (currently disabled).
#rb jb
#preflight 6139c15648e495000141820c
#rnx

[CL 17469929 by Martin Ridgers in ue5-main branch]
2021-09-09 05:37:01 -04:00
Martin Ridgers
e929a5488b Clarified that the shwon return value is in hexadecimal
#rb jb
#preflight 6139c15648e495000141820c
#rnx

[CL 17469908 by Martin Ridgers in ue5-main branch]
2021-09-09 05:36:17 -04:00
Martin Ridgers
ba0921d86f Swapped to a string-builder and adding missing TEXT() decoration around some literals
#rb jb
#preflight 6139c15648e495000141820c
#rnx

[CL 17469873 by Martin Ridgers in ue5-main branch]
2021-09-09 05:34:53 -04:00
Martin Ridgers
654b4fc909 -traceshowstore to pop up the store server in its own console window. Useful for debugging.
#rb jb
#preflight 6139c15648e495000141820c
#rnx

[CL 17469847 by Martin Ridgers in ue5-main branch]
2021-09-09 05:32:59 -04:00
aurel cordonnier
e3f7878676 Merge from Release-Engine-Test @ 17462327 to UE5/Main
This represents UE4/Main @17430120 and Dev-PerfTest @17437669

[CL 17463546 by aurel cordonnier in ue5-main branch]
2021-09-08 16:42:26 -04:00
Robert Millar
13e56208f3 Insights changes:
Make memory tag specs important so they don't get dropped before a trace connection is made.
Prevent Insights from crashing when opening a trace missing memory scopes or with missing scope specs.
Add a tooltip to symbol resolution noting the environment variable for symbol paths.
Allow Insights to capture full callstacks (including system modules) with the define UE_CALLSTACK_TRACE_FULL_CALLSTACKS or the command line argument -tracefullcallstacks on Windows.

#jira none
#rb ionut.matasaru, martin.ridgers
#robomerge[bot1] Dev-EngineMerge

[CL 17461782 by Robert Millar in ue5-main branch]
2021-09-08 15:00:25 -04:00
jason hoerner
14c85cda68 UE5: Optimizations when memory tracing is enabled: 5x perf increase with -llmcsv, 8x perf increase with -trace=memory.
#rb matt.peters johan.berg ionut.matasaru brandon.dawson jason.nadro
#rnx

Over half the perf gain was from reducing critical section locks, with the rest of the gains split between an optimization to call stack traversal (caching return address lookups in a lock free map), and enabling TLS related optimizations for MallocBinned2, which were inadvertently disabled when tracing was enabled (this also saves some locks).  Multiple lock types had to be optimized before measurable perf gains were observed, due to contention on remaining locks increasing as the first few locks were removed.  A total of 6 types of locks were removed or reduced:  AnnounceFNameTag, MallocBinned2, TagDataNameMap, Enum tag data lookup, Windows callstack trace, LLMMap.

* Added hash table utility class TGrowOnlyLockFreeHash which supports lock free reads, writes using a critical section, and no deletion.  This is intended for classes that represent caches of one-time initialized debug data used during memory tracing, which never needs to be freed.
* TGrowOnlyLockFreeHash applied to 4 sets/maps used when memory tracing is enabled, including tag data, announced tag names, call stack functions, and call stack IDs.
* Lock striping added to LLMMap.  This isn't a good candidate for a lock free approach, as it has a heavy amount of modification, and lock free approaches tend to perform worse than lock striping in that case, plus it would be a massive undertaking to make this lock free.
* A future lock to be removed would be the one in "FLowLevelAllocInfo::GetTag", protecting the "FLowLevelMemTracker::TagDatas" array, saving a couple percent more perf.
#preflight 6132a3a6bf137d00019a8c97


#ROBOMERGE-SOURCE: CL 17428762
#ROBOMERGE-BOT: (v865-17346139)

[CL 17429451 by jason hoerner in ue5-main branch]
2021-09-03 20:57:40 -04:00
Martin Ridgers
9a06a14b76 Be explicit about the "fork" argument when launching the trace store server. Previously it was implicit preventing the addition of useful help text to anyone who ran the store binary explicitly. The process is also launched "detached" so that it can be in the console sub-system without popping up conhost terminals.
#rb johan.berg
#rnx
#preflight 6130519c79ce170001a93958

[CL 17398147 by Martin Ridgers in ue5-main branch]
2021-09-02 01:44:42 -04:00
andrew firth
0ca35ab6dd [CSV] - add ExtendedUsedMB to the CSV output as a measure of the amount of extended memory currently in use.
[at]Ben.Woodhouse [at]Dave.Barrett
#rb Ben.Woodhouse
[FYI] nick.bullard
#preflight 612f969f6256b1000112e844


#ROBOMERGE-SOURCE: CL 17388268
#ROBOMERGE-BOT: (v865-17346139)

[CL 17389243 by andrew firth in ue5-main branch]
2021-09-01 13:10:13 -04:00
Martin Ridgers
be10939dc5 Specifying additional space for a traced event was only necessary if an event had an "attachment" (an arbitrary unstructured blob of data, "<< EventName.Attachment(...)"). Attachments were removed in 17080821. While the API wasn't updated at the time for transition purposes, it soon will be. (Note there is one slightly confusing element here which is that _important_ events do need to have a size specified due to the way they are buffered internally. The ThreadName events here however are not marked important).
#fyi martins.mozeiko

[CL 17365981 by Martin Ridgers in ue5-main branch]
2021-08-31 07:37:29 -04:00
martins mozeiko
bda4797854 Collect process name & id for threads in context switches capture for Insights
#rb Ionut.Matasaru

[CL 17323711 by martins mozeiko in ue5-main branch]
2021-08-26 14:51:07 -04:00
martins mozeiko
27c49fc440 Enable Windows ETW (context switches and stack sampling) and memory tracing on XSX.
Use Perf trace for context switches and stack sampling on PS4 and PS5.

#rb Ionut.Matasaru

[CL 17277312 by martins mozeiko in ue5-main branch]
2021-08-23 17:39:53 -04:00
martins mozeiko
e0dfe250ba Add missing include for non unity build
#rb Ionut.Matasaru

[CL 17141217 by martins mozeiko in ue5-main branch]
2021-08-11 15:41:51 -04:00
martins mozeiko
9ea43924d9 Enabled memory tracking with callstacks for PS4/5
Implementation details:
- PS4 & PS5 uses dwarf symbol format and uses Syms symbol resolver.
- To resolve symbols the path to folder where .self file is built currently must be specified in UE_INSIGHTS_SYMBOL_PATH env variable for Insights.
- Multiple paths can be separated by ; in this variable.
- Build for PS5 does not seem to have PLATFORM_PS5 define, I used defined(__PS5__).
- PS5 runtime collects and sends callstacks, but Syms resolver does not support dwarf v5 format yet, which is used on PS5 toolchain.

#rb none
#preflight 6112ead49c7bb10001bc63fc

[CL 17128247 by martins mozeiko in ue5-main branch]
2021-08-10 20:06:29 -04:00
charles bloom
d681814190 fix read out of bounds in Trace startup
caused crash with -stompmalloc

#rb martin.ridgers

[CL 17107919 by charles bloom in ue5-main branch]
2021-08-09 17:18:12 -04:00
Martin Ridgers
87c08c399f Corrected a spelling mistake.
[CL 17080823 by Martin Ridgers in ue5-main branch]
2021-08-06 04:44:15 -04:00
Martin Ridgers
efee157e02 It is possible that the trace store server binary legitimately doesn't exist. While it is still useful to inform the user that the store couldn't be started, it has be downgraded to a normal log message instead of a warning.
#jira UE-120128
#rnx

[CL 17067220 by Martin Ridgers in ue5-main branch]
2021-08-05 08:59:23 -04:00
Martin Ridgers
6799a4305c There is no guarantee that BinPath ends in a path separator. It is thus more robust to prefix a separator to paths appended to BinPath (two or more subsequent separators is valid and counts as a single separator).
#rnx

[CL 17065128 by Martin Ridgers in ue5-main branch]
2021-08-05 03:55:18 -04:00
aurel cordonnier
dc856801cd Merge from Release-Engine-Test @ 17059716 to UE5/Main
This represents UE4/Main @ 17030256 and Dev-PerfTest @ 17029914

[CL 17060422 by aurel cordonnier in ue5-main branch]
2021-08-04 17:46:20 -04:00
Martin Ridgers
13b3376b51 Load and platform-file tracing migrated away from attachments.
#rb cm
#rnx

[CL 17048386 by Martin Ridgers in ue5-main branch]
2021-08-04 08:07:11 -04:00
Martin Ridgers
1e2c44c08c Convert Stats and Csv tracing to use array/string type trace event fields instead of attachments
#rb cm
#rnx

[CL 17048347 by Martin Ridgers in ue5-main branch]
2021-08-04 08:03:09 -04:00
Catalin Dragoiu
df4e63ebf8 Fix non unity compile error
#rb trivial

[CL 16994685 by Catalin Dragoiu in ue5-main branch]
2021-07-29 08:53:58 -04:00
Martin Ridgers
3393e39965 Removed use of trace event attachments to send the packed CPU enter/exit timestamps in favour of an array-type field.
#rb cm
#rnx

[CL 16981291 by Martin Ridgers in ue5-main branch]
2021-07-28 08:26:51 -04:00
Martin Ridgers
40f6691fb0 Changed bookmarks trace events with an array-type field instead of using attachments.
#rb cm
#rnx

[CL 16981262 by Martin Ridgers in ue5-main branch]
2021-07-28 08:23:30 -04:00
Joe Kirchoff
2c2c2388cc Remove reference to unused header AtomicQueue/AtomicQueue.h. Make it a simple external module in case anyone needs it in the future
#rb trivial
#rnx
#preflight 61009082431c7800012fcdd3

[CL 16977624 by Joe Kirchoff in ue5-main branch]
2021-07-27 21:16:52 -04:00