Previously any events traced while Trace allocates memory to trace events into were dropped because there was nowhere to write these events. This change
splits memory allocation into two functions, one that will capture and send events when safe to do so, and one that doesn't capture events at all. For that latter, the capture is moved out to where allocating trace buffers happens so that captured events can be added to the buffer just allocated. This ensure order is maintained and that serials are ascending on a thread.
Partial buffers; prior to this change, aux-data that was larger than available space in a buffer would be fragmented across multiple buffers. Buffers that started in the middle of aux-data were marked partial as there was no way for analysis to interpret the stream from this point. As it is now possible that allocation-traced events will be written into the buffer before it's made available, each fragment is preceded with a aux-data header. This makes the stream recoverable from any buffer and negates the need for having partial buffers at all. As aux-data can now be constructed of two or more aux-data events for a single field of an event, analysis needs to be aware of this and defragment these events.
Late connect; analysis of a stream can now start with aux-data/terminal events with no corresponding owning event. This is fine; they get skipped over in DispatchEvents().
It is a deliberate choice to have a limited amount of redirected-buffer space to capture allocation-traced events. There needs to be enough space remaining in the new buffer to trace the outer event. Some platforms also have a limited amount of stack per thread.
The Writer_AddPageToPool() in Writer_InitializePool() didn't do anything and was removed.
GInitialized was set to early and subvert the reliance on static initialisation guarantees
#rnx
#jira UE-157559
#jira UE-161280
#preflight 632057f3bc40358fa290f403
[CL 21980603 by Martin Ridgers in ue5-main branch]
#ushell-cherrypick of 20644780 by Johan.Berg
#rb martin.ridgers
#rnx
#preflight 62c8260af671e8d2faccafd7
[CL 21005759 by Martin Ridgers in ue5-main branch]
Adds runtime events and analysis for generic meta data scopes. Any scope emitted on the "Metadata" writer will be analyzed and accessible in the metadata provider. Currently one built-in metadata type is provided: "Asset", which mimics LLMs implementation of asset tracking.
#rb ionut.matasaru
#robomerge EngineMerge
#preflight 62b1b35d827ccccb2cd7da0d
[CL 20755918 by Johan Berg in ue5-main branch]
Adds a new field type for trace events which contains a reference to a previously emitted event. In order to create a reference, the referenced event type needs to be declared with a new "definition" flag. The definition flag also defines type of id that is used for the reference, 8, 16, 32 and 64 bit values are supported.
#rb ionut.matasaru, martin.ridgers
#preflight 627393ea4e24dbee173d42c5
[CL 20055730 by Johan Berg in ue5-main branch]
Some rota items cannot make progress because their next event type is not known yet. They get moved to the end of the rota and the active count is decremented. There are two cases where the rota might be reprocessed from the beginning again; when an initial serial is set, and when the rota shows it has the next event. It is possible that on a subsequent pass through the rota, items that could not make progress previously might be able to this time. However, because the active count was left over from before they were excluded and OnData() would not do all the work it could possibly do.
All this only applies to old P2 traces.
#rb jb
#rnx
#preflight 620cd9a36202a22eeddb71ff
[CL 19014434 by Martin Ridgers in ue5-main branch]
If TCHAR happens to change to something other than a 16bit integer (e.g. because wchar_t is four bytes on a platform, or because the engine changes to UTF8), the analysis engine will no longer be able to read traces with char16_t-type strings.
#rb jb
#rnx
#preflight 61fd3091ed26b9fd9c7fcc0e
[CL 18862539 by Martin Ridgers in ue5-main branch]
#rb trivial
#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 18475356 via CL 18475359 via CL 18475364 via CL 18475497 via CL 18481188 via CL 18481212
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v899-18417669)
[CL 18481249 by martin ridgers in ue5-main branch]
#rb trivial
#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 18475356 via CL 18475359 via CL 18475364 via CL 18475497 via CL 18481188
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)
[CL 18481212 by martin ridgers in ue5-release-engine-test branch]
While there may be data remaining in the buffer, there might not be enough to decode another transport packet. Thus it is not correct to only consider the remaining number of bytes in the buffer as a condition of continuing to process data. Fortunately there is a demain hint that is set when someone asks for more data than a stream reader can provide. This can be used to differentiate between data remaining in the buffer, and not being able to do anything more with it.
#rb martin.ridgers, johan.berg
#rm 5.0
#ushell-cherrypick of 18267675 by Martin.Ridgers
#jira UE-134430, UE-135475
#preflight 619cf19cf70a9e92db3b69f7
[CL 18269671 by Johan Berg in ue5-main branch]
If there is more data available but none of the OnData() steps have indicated the they require more data, then OnData() can return Continue. This will allow the outer machine to make more progress, and ensure that smaller traces (where a single read can contain all sync points) makes progress.
#rb jb
#rnx
#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 18101546 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v889-18060218)
[CL 18101559 by martin ridgers in ue5-release-engine-test branch]
If there is more data available but none of the OnData() steps have indicated the they require more data, then OnData() can return Continue. This will allow the outer machine to make more progress, and ensure that smaller traces (where a single read can contain all sync points) makes progress.
#rb jb
#rnx
[CL 18101546 by Martin Ridgers in ue5-main branch]
Previously new-event and important events were collected in a pass together. However, because the new-events hadn't been processed yet, the important event stream was unintelligable until the next update. This would accellerate normal events past important ones that they might depend upon.
#rb jb
#rnx
#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17798823 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v881-17767770)
[CL 17798876 by martin ridgers in ue5-release-engine-test branch]
Previously new-event and important events were collected in a pass together. However, because the new-events hadn't been processed yet, the important event stream was unintelligable until the next update. This would accellerate normal events past important ones that they might depend upon.
#rb jb
#rnx
[CL 17798823 by Martin Ridgers in ue5-main branch]
These will help analysis know where the world was sampled at runtime in relation to the stream of data it is receiving. With these discrete points in time it is then possible to reason about gaps in events' serial numbers that can occur when late-connecting.
#rb jb
#jira ue-127376
#rnx
#preflight 615aedb7fe00590001c1440f
#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17720409 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v879-17706426)
[CL 17720457 by martin ridgers in ue5-release-engine-test branch]
These will help analysis know where the world was sampled at runtime in relation to the stream of data it is receiving. With these discrete points in time it is then possible to reason about gaps in events' serial numbers that can occur when late-connecting.
#rb jb
#jira ue-127376
#rnx
#preflight 615aedb7fe00590001c1440f
[CL 17720409 by Martin Ridgers in ue5-main branch]
