Commit Graph

71 Commits

Author SHA1 Message Date
geoff evans
0988a55b0a Trace Snapshot to File
### Features
We can now write tailing memory to a utrace file. Snapshotting while tracing to a file or socket is supported.
This change introduces a new console command: Trace.SnapshotFile. It writes the profiling data from memory into an optionally specified a utrace file.

### Testing
Run Editor with -tracefile
Run Editor with -trace, connect live with insights
Run Editor without -trace (this does generate a very small utrace since there is memory and data in the tail even though default channels are not enabled)

#rb johan.berg
#jira UE-150302,UE-150292
#preflight 6272a3213e1f2a9d3a88ee3d

#ROBOMERGE-OWNER: geoff.evans
#ROBOMERGE-AUTHOR: geoff.evans
#ROBOMERGE-SOURCE: CL 20124258 via CL 20125717 via CL 20126082
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20129506 by geoff evans in ue5-main branch]
2022-05-10 17:03:04 -04:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
martin ridgers
9688da7bbd Can't append the tail as alloc/free can get called from any thread and the tail is assumed to only be used by the worker thread (or update call). Immediately appending to the tail also desynchronises the contiguous assumption of serials within a single thread. Ultimately it's a chicken-and-the-egg problem - memory needs to be allocated to store the trace events that memory allocation traces. A redirect is kept around just in case, but its use is considered an error.
Bonus; analysis can get going quicker if a sync is added directly after the tails been sent to a new connection.
#preflight 61936d13f65efa9243c8959c

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 18203841 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v895-18170469)

[CL 18203845 by martin ridgers in ue5-release-engine-test branch]
2021-11-16 04:02:05 -05:00
martin ridgers
dea6d7fb9d Trailing whitespace.
#rnx

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17998257 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v885-17909292)

[CL 17998266 by martin ridgers in ue5-release-engine-test branch]
2021-11-01 04:37:56 -04:00
martin ridgers
0719ca133e Multi-byte character constants are non-standard
#rnx

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17998250 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v885-17909292)

[CL 17998264 by martin ridgers in ue5-release-engine-test branch]
2021-11-01 04:37:40 -04:00
martin ridgers
21bb6e7fee Fixed narrowing warnings.
#rnx

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17998132 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v885-17909292)

[CL 17998152 by martin ridgers in ue5-release-engine-test branch]
2021-11-01 04:13:24 -04:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
martin ridgers
ea47a07c3f Corrected statistics collection
Cache wasted was calculated incorrectly. Bytes-sent was missing some bytes. Memory was bumped to 64 bits just in case a 4GB accident happens.

#rb jb
#rnx
#preflight 616e6b616e12ff00011bb806

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17857934 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v883-17842818)

[CL 17857943 by martin ridgers in ue5-release-engine-test branch]
2021-10-19 04:08:40 -04:00
martin ridgers
b03efab038 Add sync points into the trace stream.
These will help analysis know where the world was sampled at runtime in relation to the stream of data it is receiving. With these discrete points in time it is then possible to reason about gaps in events' serial numbers that can occur when late-connecting.

#rb jb
#jira ue-127376
#rnx
#preflight 615aedb7fe00590001c1440f

#ROBOMERGE-AUTHOR: martin.ridgers
#ROBOMERGE-SOURCE: CL 17720409 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v879-17706426)

[CL 17720457 by martin ridgers in ue5-release-engine-test branch]
2021-10-05 09:32:12 -04:00
martin ridgers
7efe89bacc Explicitly terminate runs of aux data blocks with a well-known "aux data terminal" event. Previously a null byte was used. Each aux data block would write the null. Subsequent blocks would overwrite the prior block's null, effectively pushing the null to the end of a run aux data. However, if a different event was traced in the middle of the run it too would overwrite the null and corrupt the stream. With an explicit terminal event, structure can be recovered regardless of any multiplexing.
Serial numbers are no longer perscriptive (i.e. there can be gaps).

Normal trace events' headers are 50% smaller.

#jira ue-114532
#rb johan.berg
#preflight 6131df894f2c7c00016bf83c

#ROBOMERGE-SOURCE: CL 17419538 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v865-17346139)

[CL 17419569 by martin ridgers in ue5-release-engine-test branch]
2021-09-03 08:34:16 -04:00
martin ridgers
68899ec3ac Add a small amount of inertia when closing trace IO. The connection check is the first thing to be handled in the update meaning that any pending events prior to closing aren't sent. We can capture these pending events too if we wait one update before calling Close().
#rnx
#jira ue-109242

#ROBOMERGE-SOURCE: CL 16772364 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16772388 by martin ridgers in ue5-release-engine-test branch]
2021-06-24 09:44:12 -04:00
martin ridgers
637afa20a4 Changed the default port that Trace and its server use to 1981. This is to mitigate collision and confusion as across branches.
#rnx

#ROBOMERGE-SOURCE: CL 16704172 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16704173 by martin ridgers in ue5-release-engine-test branch]
2021-06-17 09:58:51 -04:00
martin ridgers
b499ab3d4c Removed the artifical delay that was added to support auto-connect. Previously, auto-connect was defered until we knew if we were running the editor or not. But trace would start collecting traced events and withouth a connection, would drop them. This would result in some early events being missing in auto-connect cases. We no longer wait for the editor and connect much earlier.
#rb jb
#rnx

#ROBOMERGE-SOURCE: CL 16704144 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16704159 by martin ridgers in ue5-release-engine-test branch]
2021-06-17 09:56:41 -04:00
martin ridgers
1556d3f880 Tail tracing. Maintain a tail of traced events which can be used to give historical context when late-connecting (provided there are trace channels which are enabled by default which there is not yet)
#rb jb
#rnx

#ROBOMERGE-SOURCE: CL 16296595 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v799-16237190)

[CL 16296613 by martin ridgers in ue5-release-engine-test branch]
2021-05-12 03:08:16 -04:00
Martin Ridgers
03b7ef154c Replaced some magic constants with a something more semantic.
#rb jb
#rnx

[CL 16218739 by Martin Ridgers in ue5-main branch]
2021-05-06 03:31:12 -04:00
Martin Ridgers
bd3bb6ab23 Do not return the number of bytes sent. It is no longer used anywhere
#rb jb
#rnx

[CL 16138558 by Martin Ridgers in ue5-main branch]
2021-04-28 02:58:40 -04:00
Martin Ridgers
967fb7d35e Do not try to send any prepared packets of trace data if there is no where to send the data to.
#rb jb
#rnx

[CL 16112383 by Martin Ridgers in ue5-main branch]
2021-04-26 04:03:17 -04:00
Martin Ridgers
b78da1ecce Send event descriptions out before important events. Events and important events are maintained in two separate data structures. The visibility of one with respect to the other is arbitrary from the perspective of Trace's worker thread. This could result in event descriptions being after one of their events - unrecoverable when both are in the same packet.
#rb jb
#rnx

[CL 16071395 by Martin Ridgers in ue5-main branch]
2021-04-21 07:46:59 -04:00
Martin Ridgers
f51dc81f55 Consolidated some of the duplication around building trace transport packets.
#rb jb
#rnx

[CL 16070476 by Martin Ridgers in ue5-main branch]
2021-04-21 02:33:31 -04:00
Martin Ridgers
fcc5551d6c Removed the implicit sending of data on Trace's "internal" thread in favour of being explicit where and when data should be sent in that manner.
#rb jb
#rnx

[CL 16070466 by Martin Ridgers in ue5-main branch]
2021-04-21 02:30:43 -04:00
Martin Ridgers
47783d686d Moved trace's transport enums into their own include file.
#rb jb
#rnx

[CL 15929693 by Martin Ridgers in ue5-main branch]
2021-04-06 09:35:47 -04:00
Martin Ridgers
4cd777f6cd Assert on the assumptions about the underlying transport of the trace stream. There is some code which assumes the format of packets of trace data. The change just ties them together should the transport be changed in the future.
#rb johan.berg
#rnx

[CL 15872589 by Martin Ridgers in ue5-main branch]
2021-03-31 05:24:03 -04:00
Martin Ridgers
d7af08991f Removed the default initialiser. This was originally added to register threads correct but Trace no longer makes any assumptions about thread registration.
#rb johan.berg
#rnx

[CL 15872374 by Martin Ridgers in ue5-main branch]
2021-03-31 04:25:38 -04:00
martin ridgers
ac8afd9cfe - Events that we want to capture to a buffer-redirect can't be marked as important (Important events don't use the TLS buffering path)
- The NewTrace event's UE_TRACE_LOG and Writer_SendData() were carefully ordered to make sure it is described in the stream before it is received. The SerialSync event however wasn't and the event description and traced event were in the wrong order. We'll dummy-trace both to force them to be described.
- The "SerialSync" event is used during analysis to indicate that no older events will be received and thus it is safe to derive a serial number. This scheme would fail if two successive calls to Writer_UpdateConnection() were made, which happened on boot.

#rb jb
#jira UE-109686
#rnx

#ROBOMERGE-SOURCE: CL 15602793 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v777-15581079)

[CL 15602797 by martin ridgers in ue5-main branch]
2021-03-04 05:10:09 -04:00
Marc Audy
bc88b73a29 Merge Release-Engine-Staging to Main @ CL# 15151250
Represents UE4/Main @ 15133763

[CL 15158774 by Marc Audy in ue5-main branch]
2021-01-21 16:22:06 -04:00