Commit Graph

32 Commits

Author SHA1 Message Date
ben woodhouse
3ab21db8c2 PerfReportTool 4.227.0
Make summaries and graphs collapsible. Summaries can start collapsed if specified in xml via the "collapsed" attribute.
Add a new HtmlSection helper class to assist with this.
We may later want to extend this functionality to add a top level hierarchy to the report, allowing us to have high level sections with multiple summaries in them.

[FYI] tyler.staples

[CL 30660748 by ben woodhouse in ue5-main branch]
2024-01-17 11:57:59 -05:00
ben woodhouse
477d1387ed PerfReportTool 4.226.0: Add a stat budgets summary type
This summary type sums up multiple CsvStat averages and emits the total as a SummaryTableMetric named metricName (if specified).
- Budget colors can optionally be applied to individual stats and/or the total via colourThresholds and colourThresholdsTotal nodes
- If emitStatMetrics is enabled, each stat will emit a SummaryTableMetric, named metricName/strippedStatName (with the same value as the existing CsvStatAverage element if it exists)
- Budget coloring is applied to existing CsvStatAverage elements.
- The budget for a given stat is assumed to be the first element of its colourThresholds. This value is displayed in the report
- A totalStatName attribute can be specified if a total stat already exists. If so, an "other" value will be output to the detailed summary table with the unaccounted value
- If the showTotal attribute is set to false (default: true), the total won't be shown in the detailed report

Other changes:
 - Always read all CSV stat averages, regardless of whether -readAllStats is specified. -readAllStats is only now used to determine whether to show CSV stat averages in a summary table

Example:
	<summary type="statBudgets" metricName="GTGameplay" title="Gamethread - Gameplay Budgets" hideStatPrefix="Exclusive/GameThread/" emitStatMetrics="1">
		<colourThresholdsTotal>3,4,5,6</colourThresholdsTotal>
		<colourThresholds stat="Exclusive/GameThread/Buildings">1,2,3,4</colourThresholds>
		<stats>Exclusive/GameThread/Buildings,Exclusive/GameThread/Camera,Exclusive/GameThread/Curie,Exclusive/GameThread/Network*,Exclusive/GameThread/PlayerController*,Exclusive/GameThread/WorldTickMisc,Exclusive/GameThread/FlushLatentActions,Exclusive/GameThread/Tickables,Exclusive/GameThread/SyncBodies,Exclusive/GameThread/CharPhys*,Exclusive/GameThread/Character*,Exclusive/GameThread/FortPawnTickSubsystem,Exclusive/GameThread/SignificanceManager,exclusive/gamethread/vehicle*,Exclusive/GameThread/AbilityTasks,Exclusive/GameThread/Actor*,Exclusive/GameThread/HandleRPC,Exclusive/GameThread/RepNotifies,Exclusive/GameThread/TickActors,Exclusive/GameThread/Pickups,Exclusive/GameThread/ProjectileMovement,Exclusive/GameThread/TimelineComponent,Exclusive/GameThread/TimerManager,Exclusive/GameThread/FortTrainManager</stats>
	</summary>
[FYI] andrew.ladenberger, michael.short

[CL 30329557 by ben woodhouse in ue5-main branch]
2023-12-14 15:20:48 -05:00
ben woodhouse
f860dd925d HistogramThresholds variables support + hook up to XML
[CL 30318527 by ben woodhouse in ue5-main branch]
2023-12-14 08:59:10 -05:00
ben woodhouse
2ea0faefb3 PerfReportTool 4.224.0: Adjust default colorization to make greens and reds lighter and more readable
- Add Variable support for hitch thresholds

[CL 30080549 by ben woodhouse in ue5-main branch]
2023-12-04 10:00:58 -05:00
ben woodhouse
9073333aa2 PerfReportTool 4.223.0. Budget and variable improvements
- Multiplier support for variables. Add multiplier="N" attribute to a variableset or variable
- FPSCharts : Compute frametime color thresholds automatically if MVP thresholds are specified and frametime is not
- Support empty color thresholds (useful if we want to clear via a variable)
- Add support for dumping the variables to the log for each CSV via -dumpVariables param

[CL 30060917 by ben woodhouse in ue5-main branch]
2023-12-01 19:11:26 -05:00
ben woodhouse
932a14ce02 PerfReportTool 4.210 - Variables part 1 : Add support for metadata references in ReportTypes XML. CSV Metadata references can now be specified Report xml attributes (and values in select cases) with ${meta.KEY} syntax, e.g offsetY="${meta.minimapoffsety}". Multiple variables can be matched in a single attribute if needed, eg offsetXY="${meta.minimapoffsetx},${meta.minimapoffsety}"
Support is provided for Summary classes via XElement extension helper methods, which now take an optional XmlVariableMappings param which handles the variable resolving. GetAttribute, GetSafeAttribute and GetValue support this.
Non metadata variables are also supported in theory e.g ${MyVariable}, but support for populating these does not exist yet
Various cleanup/refactoring and fixups of old code

#tests Passes Test cases with identical results (if no changes to the XML)
[FYI] tyler.staples

[CL 29912786 by ben woodhouse in ue5-main branch]
2023-11-23 14:06:40 -05:00
ben woodhouse
bf257cb06f Upgrade CSV tools to .NET core 6. Includes various required fixes for decompression, json reading, graph component hash generation and exception handling. Performance is better than .Net Framework.
[CL 29839806 by ben woodhouse in ue5-main branch]
2023-11-20 08:04:34 -05:00
kaleb morris
dec17629ad Fix bug where "minimum" stats that filter out zeros will be zero if their first value is zero
#rb ben.woodhouse, tyler.staples, matt.harris

[CL 29092162 by kaleb morris in ue5-main branch]
2023-10-25 12:21:24 -04:00
tyler staples
48925f6220 PerfReportTool - Added an "unweighted_average" formula to BoundedStatValueSummaries as weighted averages can sometimes be incorrect.
BoundedStatValueSummaries weights the stat averages by the frametimes. Normally it's not an issue to use the same value for both the value and the weight as they'll end up canceling each other out resulting in an unweighted average. However, in the case where some values are much larger than others this can skew the value resulting in a value much different from the weighted average. This is the case for server frametimes as there can be a few very large spikes that end up skewing the weights.

Since it's not trivial to detect which stats are actually "frametime" stats as they can go by different names, the simplest solution is adding a new "unweighted_average" formula. This also allows using it in other cases where it might make more sense, but these should be very rare.

#rnx
#rb andrew.ladenberger
[FYI] ben.woodhouse

[CL 26644283 by tyler staples in ue5-main branch]
2023-07-27 12:35:32 -04:00
ben woodhouse
2ba6a652ed PerfReportTool - Fix a bug with extra links templating. Some file links were not getting templated due to a change in the ExtraLink class detection logic. Make the detection explicit via a 4th optional link template param in the .links file so we can explicitly specify if a link needs template handling by specifying "linkTemplate" or "noLinkTemplate".
If the new param isn't found or "auto" is specified, detection is automatic (based on the presence of http:// or https://)

#rb tyler.staples

[CL 26296250 by ben woodhouse in ue5-main branch]
2023-06-28 15:44:22 -04:00
tyler staples
7ab16f4e82 PerfReportTool - Fix divide by 0 issue in MVP calculation.
[FYI] andrew.ladenberger

[CL 25035524 by tyler staples in ue5-main branch]
2023-04-13 20:28:01 -04:00
andrew ladenberger
b74738167a [PerfReportTool] Removed gamethread histogram from the summary table. This works better as part of the report (see HistogramSummary.cs)
#rb tyler.staples, ben.woodhouse

[CL 24343562 by andrew ladenberger in ue5-main branch]
2023-02-21 13:09:49 -05:00
jordan cristiano
4e35c288f8 Added output of per-event mvp, hitches data so we can reference it in summary tables.
#rb tyler.staples
[FYI] andrew.ladenberger, ben.woodhouse

[CL 22999920 by jordan cristiano in ue5-main branch]
2022-11-04 22:29:17 -04:00
andrew ladenberger
190bdcd52c PerfReportTool : Minor QoL updates for histograms
- Show decimals on histogram thresholds
- Shift the gamethread summary table histogram down (75%-150%)
- Renamed the gamethread summary table histogram

#rb brant.french
[FYI] tyler.staples, ben.woodhouse

[CL 22217649 by andrew ladenberger in ue5-main branch]
2022-09-27 23:53:18 -04:00
ben woodhouse
9bcde0bc1e Make "BoundedStatValuesSummary: end event appeared before the start event" non-fatal.
[CL 21500252 by ben woodhouse in ue5-main branch]
2022-08-22 21:35:02 -04:00
ben woodhouse
39009136b1 PerfReportTool 4.94 - Fix exception in HistogramSummary when SelectedStat doesn't exist. This was breaking perf reporting for devices without a GPUTime stat
[CL 21499928 by ben woodhouse in ue5-main branch]
2022-08-22 21:21:55 -04:00
andrew ladenberger
49ad5a8fc1 [PerfReportTool] Added gamethread summary stat for percent of frames over fps target at {1.25x, 1.5x, 1.75x, 2x}. This can be used in summary tables to show a histogram distrubition.
#rb ben.woodhouse
[FYI] tyler.staples, jordan.cristiano

#ROBOMERGE-AUTHOR: andrew.ladenberger
#ROBOMERGE-SOURCE: CL 21386718 via CL 21389178 via CL 21389672
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v975-21357124)

[CL 21391174 by andrew ladenberger in ue5-main branch]
2022-08-15 14:54:21 -04:00
andrew ladenberger
679b0004c6 PerfReportTool - Updated the Histogram summary table
- New option to suppress the averages table
- Reversed from under to over budget
- Removed the hitch table since the tool already has a seperate hitch summary

#rb Ben.Woodhouse
[FYI] Tyler.Staples

#ROBOMERGE-AUTHOR: andrew.ladenberger
#ROBOMERGE-SOURCE: CL 21358870 via CL 21358876 via CL 21358887
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21360773 by andrew ladenberger in ue5-main branch]
2022-08-12 15:06:33 -04:00
tyler staples
740667d3d1 Updated PerfReportTool to treat extra links starting with '/' as links instead of link templates.
#rb ben.woodhouse

#ROBOMERGE-AUTHOR: tyler.staples
#ROBOMERGE-SOURCE: CL 20984783 via CL 20984818 via CL 20984849
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 20988899 by tyler staples in ue5-main branch]
2022-07-07 14:38:12 -04:00
ben woodhouse
5fbb76f7c2 PerfReportTool 4.76 : Add support for unstripped stats in FPSCharts.
- FPSChart stats can now be tagged as unstripped. Add "unstripped" in the stat bracket parameters, space-separated to enable, e.g MemFreeMB (min unstripped).
- Simplify column handling code
- Don't generate CsvToSvg commandline args if we're not running as a separate process

Note: The unstripped parameter does not affect the column name. E.g MemFreeMB(Min Unstripped) will be output as MemFreeMB Min.

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 20315540 via CL 20315544 via CL 20315547
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v948-20297126)

[CL 20315662 by ben woodhouse in ue5-main branch]
2022-05-22 16:34:09 -04:00
ben woodhouse
45fdd234a9 PerfReportTool 4.75 - Add logging around map image handling
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19862730 via CL 19866748 via CL 19866963
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v940-19807014)

[CL 19871403 by ben woodhouse in ue5-main branch]
2022-04-22 16:06:16 -04:00
ben woodhouse
cf8c132bde PerfReportTool 4.74
- Link templates are only added for extra relative links. Links starting with http:// or https:// are not replaced

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19828731 via CL 19828779 via CL 19828794
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v940-19807014)

[CL 19835207 by ben woodhouse in ue5-main branch]
2022-04-20 14:43:25 -04:00
ben woodhouse
2709ab9660 Shrink map summary images to the destination size rather than just copying
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19782302 via CL 19782307 via CL 19782309
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803417 by ben woodhouse in ue5-main branch]
2022-04-19 00:19:45 -04:00
ben woodhouse
8f29d53bc6 PerfReportTool updates to support new PerfReportServer
PerfReportTool 4.71
 - Add support for summary table link templates that can be replaced later (e.g by a web server). E.g {LinkTemplate:Csv:<CSVID>}. Pass -linkTemplates to enable. This works for Csv and Report links, as well as ExtraLinks
 - Add -csvList and -PRCList args for specifying a list of PRCs or CSVs
 - Add -summaryTableXML arg for specifying an additional XML file containing summary tables
 - Summary table titles can now be specified with -summaryTitle instead of this being automatic

All CSV tools:
 - Add response file commandline support via -response=<filename>


#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19686909 via CL 19686925 via CL 19686934
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19687527 by ben woodhouse in ue5-main branch]
2022-04-08 11:08:05 -04:00
ben woodhouse
cce1015f02 PerfReportTool 4.68 - Fix peak summary not emitting SummaryTableData when detailed reports are disabled
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 18344863 via CL 18344911 via CL 18344933 via CL 18348483 via CL 18348520
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18348589 by ben woodhouse in ue5-release-engine-test branch]
2021-12-01 21:14:47 -05:00