- TraceAnalysis: Added debug functionality for trace transport and trace protocol analysis. See TraceAnalysisDebug.h for the available debug options (UE_TRACE_ANALYSIS_DEBUG_API, UE_TRACE_ANALYSIS_DEBUG, UE_TRACE_ANALYSIS_DEBUG_LOG, UE_TRACE_ANALYSIS_DEBUG_LOG_IMPL, UE_TRACE_ANALYSIS_DEBUG_LEVEL).
- TraceAnalysis: Changed the limit for accumulated too much data on a single thread from 512 MiB to 2000 MB. Also implemented better code to skip serials when limit is encountered.
- TraceAnalysis: Implemented workaround for corrupted traces (events with aux blocks missing the "aux terminal").
- TraceAnalysis: Fixed EventData.GetArray<>() (to be safe to be called multiple times).
#rb Johan.Berg, Catalin.Dragoiu
[FYI] Martin.Ridgers
[CL 29268256 by ionut matasaru in ue5-main branch]
Usage of MessageLog in non-ui threads is tricky and need to be carefully managed to avoid flushing memory on the wrong thread. With this change direct usage of MessageLog in the analysis engine is removed and replaced with a callback system which allows the user of TraceAnalysis to implement display of the messages themselves. In the case of InsightsManager this means queuing up the messages in analysis session and regularly polled from the UI thread.
#rb ionut.matasaru
#jira UE-185528
[CL 26086861 by Johan Berg in ue5-main branch]
The main premise of this change is to order the heap of pending serialised events by distance from the next expected serial instead of in respect to each other. By using an unsigned comparison, wrapped serials that would otherwise appear on the negative side of the expected serial, in fact sort to the positive side. This allows the use of the full serial space as wraps no longer need to be calculated when sorting.
#rb jb
#rnx
#preflight 63caa0e6894b6403149bf5c0
[CL 23789248 by Martin Ridgers in ue5-main branch]
We have previously relied on UE_LOG messages to indicate problems in analysis. This change converts those log messages to use MessageLog functionality which is routed to a visual log representation inside the application (but also the log file).
Additionally adds functionality to show the Insights log windows when ever (new) errors are encountered.
#rb ionut.matasaru
#jira UE-154022
#preflight 63c91aa80225f00e14758d19
[CL 23771338 by Johan Berg in ue5-main branch]
- Added workaround for large utrace streams with corrupted serial sync events (detects when too much data is buffered on one thread and stops analysis automatically).
- Changed the read buffers to increase their size in power of two steps.
#rb Martin.Ridgers
#preflight 639996388c5081ee9a857e57
[CL 23510588 by ionut matasaru in ue5-main branch]
Previously any events traced while Trace allocates memory to trace events into were dropped because there was nowhere to write these events. This change
splits memory allocation into two functions, one that will capture and send events when safe to do so, and one that doesn't capture events at all. For that latter, the capture is moved out to where allocating trace buffers happens so that captured events can be added to the buffer just allocated. This ensure order is maintained and that serials are ascending on a thread.
Partial buffers; prior to this change, aux-data that was larger than available space in a buffer would be fragmented across multiple buffers. Buffers that started in the middle of aux-data were marked partial as there was no way for analysis to interpret the stream from this point. As it is now possible that allocation-traced events will be written into the buffer before it's made available, each fragment is preceded with a aux-data header. This makes the stream recoverable from any buffer and negates the need for having partial buffers at all. As aux-data can now be constructed of two or more aux-data events for a single field of an event, analysis needs to be aware of this and defragment these events.
Late connect; analysis of a stream can now start with aux-data/terminal events with no corresponding owning event. This is fine; they get skipped over in DispatchEvents().
It is a deliberate choice to have a limited amount of redirected-buffer space to capture allocation-traced events. There needs to be enough space remaining in the new buffer to trace the outer event. Some platforms also have a limited amount of stack per thread.
The Writer_AddPageToPool() in Writer_InitializePool() didn't do anything and was removed.
GInitialized was set to early and subvert the reliance on static initialisation guarantees
#rnx
#jira UE-157559
#jira UE-161280
#preflight 632057f3bc40358fa290f403
[CL 21980603 by Martin Ridgers in ue5-main branch]
#ushell-cherrypick of 20644780 by Johan.Berg
#rb martin.ridgers
#rnx
#preflight 62c8260af671e8d2faccafd7
[CL 21005759 by Martin Ridgers in ue5-main branch]
Adds runtime events and analysis for generic meta data scopes. Any scope emitted on the "Metadata" writer will be analyzed and accessible in the metadata provider. Currently one built-in metadata type is provided: "Asset", which mimics LLMs implementation of asset tracking.
#rb ionut.matasaru
#robomerge EngineMerge
#preflight 62b1b35d827ccccb2cd7da0d
[CL 20755918 by Johan Berg in ue5-main branch]
Adds a new field type for trace events which contains a reference to a previously emitted event. In order to create a reference, the referenced event type needs to be declared with a new "definition" flag. The definition flag also defines type of id that is used for the reference, 8, 16, 32 and 64 bit values are supported.
#rb ionut.matasaru, martin.ridgers
#preflight 627393ea4e24dbee173d42c5
[CL 20055730 by Johan Berg in ue5-main branch]
Some rota items cannot make progress because their next event type is not known yet. They get moved to the end of the rota and the active count is decremented. There are two cases where the rota might be reprocessed from the beginning again; when an initial serial is set, and when the rota shows it has the next event. It is possible that on a subsequent pass through the rota, items that could not make progress previously might be able to this time. However, because the active count was left over from before they were excluded and OnData() would not do all the work it could possibly do.
All this only applies to old P2 traces.
#rb jb
#rnx
#preflight 620cd9a36202a22eeddb71ff
[CL 19014434 by Martin Ridgers in ue5-main branch]
If TCHAR happens to change to something other than a 16bit integer (e.g. because wchar_t is four bytes on a platform, or because the engine changes to UTF8), the analysis engine will no longer be able to read traces with char16_t-type strings.
#rb jb
#rnx
#preflight 61fd3091ed26b9fd9c7fcc0e
[CL 18862539 by Martin Ridgers in ue5-main branch]
