* Adds logic to sort tags into correct hierarchy in analysis, regardless if the tag was declared as an LLMTag enum, name based tag or declared before it's parent has been declared.
* Add new class to break down memory allocs into hiearchical tags tree.
#rb ionit.matasaru
#preflight 623866cd88538cd45ec5ccc1
[CL 19450624 by Johan Berg in ue5-main branch]
- Improved performance of memory analysis (processing of live allocations in AllocationsProvider) with up to 50% (i.e. twice as fast). This is achived by using a better hash for TMap for long living allocations. Also switched the short living allocations to use only the linked list, with a reduced size (64 allocs).
- Delayed initialization of SystemMemory heap to when the allocations provider inits (to save memory usage when traces does not have mem alloc events).
- Added option to enable an initial "reserve" number for TMap (one used for long living allocations). Benchmarks shows further performance increase when using large reserve numbers, but mem cost becomes too high. So kept this 0 (unset) by default.
- Added error checking for very large mem traces (when total number of mem events could exceed 32 bit limit).
Benchmarks:
* analysis of a 32s session: from 17.6s to 8.3s (-53%)
* analysis of a 7min session: from 88s to 46s (-48%)
#jira UE-135890
#rb Johan.Berg
#preflight 620fab43a96c65b0d81a9566
[CL 19070177 by ionut matasaru in ue5-main branch]
- Memory Insights: Added analysis support for "free callstack" for an allocation (i.e. callstack when an allocation if freed).
- Memory Insights: Switched CallstackId to 32 bit and moved the resolving of "id to pointer" outside of mem queries. This allows us to add the second callstack id and also ids for future metadata without increasing the size of FAllocationItem struct (from AllocationProvider).
- Memory Insights: Refactored code re ThreadId and Tracker passed from AllocationAnalysis to AllocationProvider.
- Memory Insights: Refactored code re CallstackProvider to map a CallstackId directly as an index into array of callstacks.
#rb Johan.Berg
#preflight 6201460a6773a3612898c4de
[CL 18886419 by ionut matasaru in ue5-main branch]
Introduced a type alias for Tag type id, abstracting how we represent tags internally. This doesn't represent a a functional change, as in practise existing instrumentation only emitted postive values and previously uint32 was used for storage.
#rb none
#ROBOMERGE-AUTHOR: johan.berg
#ROBOMERGE-SOURCE: CL 18308342 in //UE5/Release-5.0/... via CL 18308344
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)
[CL 18308390 by johan berg in ue5-release-engine-test branch]
Make memory tag specs important so they don't get dropped before a trace connection is made.
Prevent Insights from crashing when opening a trace missing memory scopes or with missing scope specs.
Add a tooltip to symbol resolution noting the environment variable for symbol paths.
Allow Insights to capture full callstacks (including system modules) with the define UE_CALLSTACK_TRACE_FULL_CALLSTACKS or the command line argument -tracefullcallstacks on Windows.
#jira none
#rb ionut.matasaru, martin.ridgers
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 17461782 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[bot1]: Dev-EngineMerge
[CL 17461823 by robert millar in ue5-release-engine-test branch]
- TraceServices: Changed ReadNetProfilerProvider, ReadMemoryProvider, ReadDiagnosticsProvider to return a pointer to the respective provider (instead of a reference). Can return nullptr. The UI should not assume the provider exists.
- TraceServices: Added GetNetProfilerProviderName, GetMemoryProviderName, GetDiagnosticsProviderName, GetAllocationsProviderName to return name of respective provider.
- TraceServices: Added ReadModuleProvider + GetModuleProviderName, ReadCallstacksProvider + GetCallstacksProviderName to access respective providers.
- TraceServices: Added a default empty implementation to IModule for GenerateReports, GetLoggers and GetCommandLineArgument.
- TraceServices: Changed FAnalysisService::StartAnalysis to not create default providers for Memory (LLM Stats), NetProfiler and Diagnostics, but to allow the respective modules (FMemoryModule, FNetProfilerModule, FDiagnosticsModule) to create the providers.
- TraceInsights: Fixed UI code to check if the Memory, NetProfiler and Diagnostics provider are available.
#rb Catalin.Dragoiu, Johan.Berg
#ROBOMERGE-SOURCE: CL 16967698 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)
[CL 16967704 by ionut matasaru in ue5-release-engine-test branch]
- Memory: Improved speed of live allocs matching in memory analysis.
- Memory: Fixed N/A callstacks in a query made before analysis is completed.
#jira UE-110125
#rb Catalin.Dragoiu
[CL 15736769 by ionut matasaru in ue5-main branch]
Realloc scopes were not applied. Additionally we extracted incorrect information from the event and constructed the combined thread and tracker information wrong.
#rb ionut.matasaru
#jira UE-110014
[CL 15720460 by Johan Berg in ue5-main branch]
- Implemented mem alloc queries also for live allocs.
- Implemented analysis for llm tags associated with allocations (based on initial work by Johan.Berg).
- Changed granularity of memory allocations timelines to 0.1ms (from 1ms).
- Fixed allotment of allocations in SbTree's offsetted cells.
- Fixed queries to handle also cases where multiple cells have exact same timestamps.
- Added more debug/validation code for AllocationsProvider.
#rb Catalin.Dragoiu
[CL 14938990 by ionut matasaru in ue5-main branch]
- Fixed the Allocations Analysis to process the updated alloc trace events.
- Added timelines in the Allocations Provider with variation for several stats based on 1ms sample:
* MinTotalAllocatedMemory
* MaxTotalAllocatedMemory
* MinLiveAllocationCount
* MaxLiveAllocationCount
* AllocEventCount
* FreeEventCount
- Added graph tracks and graph series for all above timelines in Memory Insights.
#rb Catalin.Dragoiu
#jira UECORE-635
[CL 14853556 by ionut matasaru in ue5-main branch]
- Added detection for AllocationsProvider availability to know when to enable the Memory Insights tab (in addition to detecting availability of the LLM tags memory provider).
- Simplified the API for allocation queries (using A, B, C, D time markers).
- Added initial AllocationsAnalysis that processes the allocation trace events and delegates them to AllocationsProvider.
- Added initial AllocationsProvider. Uses a simple map for live allocations.
- Added FSbTree ("SBIF with offsetted cells" data structure for storing the retired allocations in AllocationsProvider).
- Added async allocation queries (based on A, B, C, D time markers). Currently it uses a single TaskGraph task.
- Added "LogTraceServices" log category for general purpose logging in TraceServices module.
#rb Catalin.Dragoiu
[CL 14813195 by ionut matasaru in ue5-main branch]
