Change consist of only forward declaration and additional includes
#preflight 63789c1de30d438849c48188
#rb none
[CL 23218412 by henrik karlsson in ue5-main branch]
Most of the warnings were about unaligned memory access. Such access is not a problem on platforms we care about (an exception being 32 bit ARM and int64's, but we memcpy those already). As such we can use a memcpy() to remove the undefined behaviour and still have the compiler output decent instructions (in >= Development variants). This unfortunately semantically unlinks the structs, but comments have been left should anyone go splunking.
New-Event event writes are already on two-byte boundaries or less, so aligning them up to two bytes in size maintains alignment.
Lastly, UBSAN warns about how TraceLog plays hard and fast with the rules to cast arbitrary pointers to certain types in order to implement the UE_TRACE_LOG() API. We can use C++17's std::launder to communicate intent to the compiler.
#rb cm
#rnx
#preflight 635ba2224710dd6af89520cd
[CL 22826416 by Martin Ridgers in ue5-main branch]
Doing so keeps TSAN happy. This isn't really thread safe as there's no synchronisation. However, the only scenario that this would be a problem is when someone is calling some combination of send/write/stop in very quick succession. And if that's going on, something else is very wrong.
#rb dc
#rnx
#preflight 63564a329e14ee3c7940f270
[CL 22718563 by Martin Ridgers in ue5-main branch]
NextBuffer is used both to maintain a list of available buffers and to traverse threads' active buffers to collect and send traced data. It was synchronised by other means (and in some rare instances allowed to race harmlessly). TSAN couldn't follow this. Not all warnings were false positives however, there was a risk that loads/stores could be reordered when making a buffer visible to Trace's worker thread.
#rb dc
#rnx
#preflight 63564c4d9e14ee3c7941213b
[CL 22718470 by Martin Ridgers in ue5-main branch]
TSAN doesn't realise that the function-scoped static takes care of the lazy-initialisation's thread safety. But this is not a hot path, nor is synchronisation of GInit critical, so we can use relaxed load/stores and TSAN is a happy bunny.
#rb dc
#rnx
#ushell-unshelve of 22562441
#preflight 634e502bc02c14bba03557b2
[CL 22594979 by Martin Ridgers in ue5-main branch]
Trace uses a thread local object and it destructor as a means to collect buffers when threads end. This usually happens lazily, and usually when fetching a new identified for a thread. On some platforms can result in an allocation and if memory tracing was on, its events would end up in the redirect buffer as the correct buffer hadn't yet been set. But because the redirect had already been memcpy'd, events would get lost.
#rb im
#rnx
#jira UE-157559
#preflight 633a7e975c2225fe5f32a5f0
[CL 22295530 by martin ridgers in ue5-main branch]
After collecting important events, Trace will free the buffers used. If memory tracing is enabled then the Writer_Free() call will immediately send some trace events. If a connection already exists, and this is the first trip through the update loop, then there were some prior events traced to set up the thread. These prior events have lower serial numbers but haven't yet been collected and sent out. Thus Free() ends up sending events too early. There is actually no need to redirect here; Trace doesn't shut itself down, but instead leaves it to the kernel to clean up after the process exits. This is so it can memory events can be traced right to the very end of the process, including during static-exit.
#rb im
#rnx
#jira ue-157559
#preflight 633a7e7fc37844870ab7e3fc
[CL 22295528 by martin ridgers in ue5-main branch]
The PS4 standard C++ library uses an atexit() style call to register destructors of thread-local objects. The implementation takes a lock. This can easily deadlock if memory tracing is enabled as memory tracing will trace when allocating unrelated TLS-related memory. This change uses an alternative scheme to get a notification when a thread terminates.
#jira UE-163770
#rb im
#rnx
#preflight 632ab930f45b523a6345570a
[CL 22112957 by martin ridgers in ue5-main branch]
It is only when allocating buffers to store normal trace events that care needs to be taken to capture events and blit them to the appropriate place. This is already in place in Writer_NextBuffer(). This also handles cases where traced events are artifacts of Trace allocating memory for other purposes. So the redirect removed by this change was incorrect. Under the correct conditions, it would accelerate events ahead of others due to the SendData() call and result in the incorrect ordering events being sent across the wire.
#rb im
#jira UE-163770
#rnx
#preflight 63296e009840225da24c032e
[CL 22090242 by martin ridgers in ue5-main branch]
The redirect buffer was a little too small to contain the callstack that may get traced. Bumping the size isn't the ideal solution but suitable for the now. Alternatives would be to delta compress callstack events (only an option on monolithic builds), take smaller backtraces for trace-related allocations, eliminate redundancy in addresses (i.e. a 32 bit encoding scheme), or possibly segmented deduplication.
#rb im
#jira UE-163770
#rnx
#preflight 63242bb274361465144c7c4d
[CL 22050312 by martin ridgers in ue5-main branch]
The size of the segment of aux-data to memcpy into the buffer incorrectly included the size of the aux-header. This would result in writing aux-data off the end of the overflow and into the low-order bits of the timestamp bias. This would inadvertantly cause incorrect, sometimes negative, timestamps for scoped events.
#rb im
#jira UE-163769
#rnx
#preflight 632325d3d5013ce29f43854c
[CL 22035676 by martin ridgers in ue5-main branch]
* Added trace file created callback: Useful to gather all trace files created during execution
* Added config option to disable or resize tail buffers via [Trace.Config]:TailSizeBytes
* Added config option for worker thread sleep time via [Trace.Config]:SleepTimeInMS
* Fixed init code so it never creates a workerthread in builds that don't support multithreading
#rb Martin.Ridgets, Ionut.Matasaru
[CL 22013173 by louisphilippe seguin in ue5-main branch]
Previously any events traced while Trace allocates memory to trace events into were dropped because there was nowhere to write these events. This change
splits memory allocation into two functions, one that will capture and send events when safe to do so, and one that doesn't capture events at all. For that latter, the capture is moved out to where allocating trace buffers happens so that captured events can be added to the buffer just allocated. This ensure order is maintained and that serials are ascending on a thread.
Partial buffers; prior to this change, aux-data that was larger than available space in a buffer would be fragmented across multiple buffers. Buffers that started in the middle of aux-data were marked partial as there was no way for analysis to interpret the stream from this point. As it is now possible that allocation-traced events will be written into the buffer before it's made available, each fragment is preceded with a aux-data header. This makes the stream recoverable from any buffer and negates the need for having partial buffers at all. As aux-data can now be constructed of two or more aux-data events for a single field of an event, analysis needs to be aware of this and defragment these events.
Late connect; analysis of a stream can now start with aux-data/terminal events with no corresponding owning event. This is fine; they get skipped over in DispatchEvents().
It is a deliberate choice to have a limited amount of redirected-buffer space to capture allocation-traced events. There needs to be enough space remaining in the new buffer to trace the outer event. Some platforms also have a limited amount of stack per thread.
The Writer_AddPageToPool() in Writer_InitializePool() didn't do anything and was removed.
GInitialized was set to early and subvert the reliance on static initialisation guarantees
#rnx
#jira UE-157559
#jira UE-161280
#preflight 632057f3bc40358fa290f403
[CL 21980603 by Martin Ridgers in ue5-main branch]
