Commit Graph

40 Commits

Author SHA1 Message Date
Ryan Hummer
4af2fd066d Updating Dev-Release-5.5 from Main at CL #36144969
#okforversepublic

[CL 36146571 by Ryan Hummer in Dev-5.5 branch]
2024-09-10 10:26:02 -04:00
ben woodhouse
e7586403de Make padding trailing digits optional using the -sortTrailingDigitsAsNumeric param since it can cause issues
[FYI] tom.holmes

[CL 34062555 by ben woodhouse in ue5-main branch]
2024-06-03 07:29:14 -04:00
ben woodhouse
143b9b0331 PerfReportTool 4.237.0 - Make summary table sorting smarter. If a column with string values includes an integer suffix, left pad with zeroes for sorting purposes
[CL 33974852 by ben woodhouse in ue5-main branch]
2024-05-29 11:20:53 -04:00
ben woodhouse
d4d2df2b83 PerfReportTool 4.233.0 : Unify wildcard handling for summary tables so it correctly matches mid-string wildcards. Also optimize using a binary search
[CL 31979848 by ben woodhouse in ue5-main branch]
2024-03-03 10:27:37 -05:00
ben woodhouse
1f59f63a96 PerfReportTool 4.232.0: Add support for stat type qualifiers in summary tables so we can identify stats by their type to avoid ambiguity, e.g [meta]deviceprofile. Prefixes are :
[metric] SummaryTableMetric
[csv] CsvStatAverage
[meta] CsvMetadata
[toolmeta] ToolMetadata

[CL 31978791 by ben woodhouse in ue5-main branch]
2024-03-02 20:24:47 -05:00
ben woodhouse
90a828c869 PerfReportTool: Also output the elementType in the header tooltip (E.g CsvMetadata, SummaryTableMetric etc)
[CL 31599556 by ben woodhouse in ue5-main branch]
2024-02-17 07:08:25 -05:00
ben woodhouse
b8f8c1b7b2 PerfReportTool: When stat names are remapped, display the original name as a tooltip
[CL 31578316 by ben woodhouse in ue5-main branch]
2024-02-16 15:42:57 -05:00
ben woodhouse
6b06bd7eaa PerfReportTool 4.230.0 - Add support for variables which can be used outside of individual CSV reporting. Any variable declared in a variableSet without a metadata query can be used in this way. This is useful if we want to access variables in SummaryTable definitions for example. Also strip whitespace in summary table row sort and filter lists
[CL 30955693 by ben woodhouse in ue5-main branch]
2024-01-27 11:28:43 -05:00
ben woodhouse
3ab21db8c2 PerfReportTool 4.227.0
Make summaries and graphs collapsible. Summaries can start collapsed if specified in xml via the "collapsed" attribute.
Add a new HtmlSection helper class to assist with this.
We may later want to extend this functionality to add a top level hierarchy to the report, allowing us to have high level sections with multiple summaries in them.

[FYI] tyler.staples

[CL 30660748 by ben woodhouse in ue5-main branch]
2024-01-17 11:57:59 -05:00
ben woodhouse
146dc093bb PerfReportTool 4.210.1 : fix an exception when startTimestamp is not set for some rows
[CL 29915928 by ben woodhouse in ue5-main branch]
2023-11-23 22:04:05 -05:00
ben woodhouse
932a14ce02 PerfReportTool 4.210 - Variables part 1 : Add support for metadata references in ReportTypes XML. CSV Metadata references can now be specified Report xml attributes (and values in select cases) with ${meta.KEY} syntax, e.g offsetY="${meta.minimapoffsety}". Multiple variables can be matched in a single attribute if needed, eg offsetXY="${meta.minimapoffsetx},${meta.minimapoffsety}"
Support is provided for Summary classes via XElement extension helper methods, which now take an optional XmlVariableMappings param which handles the variable resolving. GetAttribute, GetSafeAttribute and GetValue support this.
Non metadata variables are also supported in theory e.g ${MyVariable}, but support for populating these does not exist yet
Various cleanup/refactoring and fixups of old code

#tests Passes Test cases with identical results (if no changes to the XML)
[FYI] tyler.staples

[CL 29912786 by ben woodhouse in ue5-main branch]
2023-11-23 14:06:40 -05:00
ben woodhouse
31edc9c43b When sorting columns by diff, keep CSV stats of the same category together. Do an addional stable sort by stat category after the initial sort by diff
[FYI] tyler.staples

[CL 25990671 by ben woodhouse in ue5-main branch]
2023-06-14 15:34:16 -04:00
ben woodhouse
ded3c14346 PerfReportTool 4.105.0: Add support for sorting columns by diff and also filtering out columns based on a diff value threshold.
Adds the following params:
  -sortColumnsByDiff : sorts columns by the max of its diff scores (use with -addDiffRows)
  Note: Diff score corresponds to the value of a column's diff row; the sign is reversed if LowIsBad

  -columnDiffDisplayThreshold <value> : if specified, hides columns with max abs diff value below this threshold

Also: added some helpers to summary tables to find aggregate columns . Added a isCountColumn accessor to avoid string compares

[CL 25982885 by ben woodhouse in ue5-main branch]
2023-06-14 12:34:40 -04:00
ben woodhouse
6785e92518 PerfReportTool - avoid div/0 in autocolorization
[FYI] tyler.staples

[CL 25952964 by ben woodhouse in ue5-main branch]
2023-06-13 11:27:26 -04:00
ben woodhouse
feec4458d9 PerfReportTool 4.104 - Adjust auto-colorization to reduce the color difference when the difference between min/max is very small. Also add a threshold to disable colorization altogether.
[FYI] tyler.staples,tom.holmes

[CL 25950988 by ben woodhouse in ue5-main branch]
2023-06-13 10:20:08 -04:00
tyler staples
d2ddfa358a PerfReportTool - Add support for date formats and disabling column word wrap.
#rb ben.woodhouse

[CL 24905945 by tyler staples in ue5-main branch]
2023-04-03 22:30:12 -04:00
ben woodhouse
0e7d1d7ea0 PerfReportTool - Add -summaryTableXmlRowSortAppend arg to allow appending values to a summary table's row sort list via the commandline
[CL 24136498 by ben woodhouse in ue5-main branch]
2023-02-11 12:25:29 -05:00
ben woodhouse
f7cd8f51d5 PerfReportTool: add -summaryTableXmlAppend arg for appending stats/metrics to the end of a summary table's filter list via the commandline. This allows us to append multiple stats (-summaryTableXmlSubst only supports one stat per substitution)
[CL 24092968 by ben woodhouse in ue5-main branch]
2023-02-09 00:36:36 -05:00
tyler staples
3b43312f37 PerfReportTool - Add support for regression and other column filters.
+ Additional column filters can now be created by implementing ISummaryTableColumnFilter and adding it to the list in Program.MakeAdditionalColumnFilters.
+ Moved stat threshold checks into an additional filter.
+ Added a column filter to hide metadata columns.
+ Added a column filter to only show columns where the most recent build has regressed by more than a given threshold.
+ You can now use 'debugShowFilteredColumns' to have columns be grayed out instead of removed. Hovering over them will show the reason they were filtered out (which filter and the reason). This is for debugging purposes and sanity checks to make sure you're not missing important information, especially when iterating on new filters. Some additional settings were added to SummaryTableColumn to support this.
+ Filters can also mark individual cells as 'invalid' to indicate they are not taken into consideration for that filter. These are grayed out with a tooltip when debugShowFilteredColumns is enabled. Currently this does not work in collated views.
+ Added the ability to set tooltips for column headers.
+ Added the ability to add a color modifier for the entire column or individual cells.

#rb ben.woodhouse

[CL 22890943 by tyler staples in ue5-main branch]
2022-11-01 16:30:42 -04:00
tyler staples
b02c452ae8 PerfReportTool - Update budget colour scheme to match auto as it's much easier on the eyes.
#rb ben.woodhouse

[CL 22798511 by tyler staples in ue5-main branch]
2022-10-26 19:10:45 -04:00
tyler staples
c94b11439d PerfReportTool - Add support for colourThreshold overrides in summaryTableColumnFormatInfo.
This allows setting colourThresholds globally for an entire column, overriding other settings.

Also added support for specifiying the actual colours and if they should be lerped between or not.
Note: lerpColours isn't currently serialized to prc files.

#rb ben.woodhouse

[CL 22712141 by tyler staples in ue5-main branch]
2022-10-22 15:34:08 -04:00
tyler staples
d49dd7aded PerfReportTool - Add support for showing value buckets in columns.
This allows you to define a list of value thresholds for a column and the bucket names that correspond with each threshold. When present the corresponding bucket name will be displayed in the column instead of/with the value.

#rb ben.woodhouse

[CL 22670762 by tyler staples in ue5-main branch]
2022-10-20 16:47:38 -04:00
tyler staples
ed4771efaa Fixed inconsistencies with -colorizeTable and it not correctly overriding the colorMode set in the summary table xml.
+ 'default' now only exists as an argument and not an actual state as default means use what is defined in the summary table xml file.
+ Removed the 'Default' TableColorizeMode value and added a 'Budget' value which represents budget coloring.

#rb ben.woodhouse

[CL 22662548 by tyler staples in ue5-main branch]
2022-10-20 13:44:01 -04:00
ben woodhouse
fe8e2509aa PerfReportTool 4.94.2 - Fix diff row bug which caused an extra all-zero row to be created for every diff row added. This was breakng automatic colorization, causing all previous rows to be red
[CL 21500256 by ben woodhouse in ue5-main branch]
2022-08-22 21:35:10 -04:00
ben woodhouse
a3c3ea66e9 PerfReportTool 4.91 - replace autoColorize bool xml param with colorizeMode, which can be default, auto or off (new option). Add equivalent commandline param (-colorizeTable)
- Adding a diff row no longer disables colorization (unless colorizeMode is explicitly set to off)
- More formatting improvements for transposed tables

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 21238557 via CL 21238558 via CL 21239577
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21239660 by ben woodhouse in ue5-main branch]
2022-07-26 17:00:58 -04:00