Commit Graph

165 Commits

Author SHA1 Message Date
ben woodhouse
8d91b3258f PerfReportTool 4.108.0 + CsvToSvg: -maxAutoMaxY support for graphs + other fixes:
- Add maxAutoMaxY param to graphs and csvToSVG params which allows us to clamp the max automatic Y to a particular value. Also:
- Fix some naming clashes when there are mulltiple graphs on a page. This was preventing us from having graphs with different sizes in a perf report
- Add -embedGraphCommandline arg to help debugging
[FYI] tyler.staples

[CL 29745624 by ben woodhouse in ue5-main branch]
2023-11-15 09:55:24 -05:00
ola olsson
36b8421f1f Added the ability to pass in a list of custom legend entries that match the list of csv files & increased the maxy limit to 8M.
#rb ben.woodhouse

[CL 29743496 by ola olsson in ue5-main branch]
2023-11-15 07:46:27 -05:00
ben woodhouse
527126fd2c Rebuild PerfReportTool to fix an exception, and bump version to 4.107.2 (cs change was checked in but exe hadn't been rebuilt)
[CL 29189970 by ben woodhouse in ue5-main branch]
2023-10-27 14:09:15 -04:00
kaleb morris
dec17629ad Fix bug where "minimum" stats that filter out zeros will be zero if their first value is zero
#rb ben.woodhouse, tyler.staples, matt.harris

[CL 29092162 by kaleb morris in ue5-main branch]
2023-10-25 12:21:24 -04:00
ben woodhouse
fcbbb0d785 CsvToSVG 2.58 (and PerfReportTool 4.107.1) : Fix graphing issue with highlightEventRegion where begin can't be null. This allows us to grey out regions at the start of matches
[CL 28945269 by ben woodhouse in ue5-main branch]
2023-10-19 20:14:08 -04:00
tyler staples
342b522b5f PerfReportTool - Fix a crash when reading partial csvs (just the end so we can read metadata).
#rnx
[FYI] ben.woodhouse

[CL 26922403 by tyler staples in ue5-main branch]
2023-08-08 13:28:43 -04:00
tyler staples
48925f6220 PerfReportTool - Added an "unweighted_average" formula to BoundedStatValueSummaries as weighted averages can sometimes be incorrect.
BoundedStatValueSummaries weights the stat averages by the frametimes. Normally it's not an issue to use the same value for both the value and the weight as they'll end up canceling each other out resulting in an unweighted average. However, in the case where some values are much larger than others this can skew the value resulting in a value much different from the weighted average. This is the case for server frametimes as there can be a few very large spikes that end up skewing the weights.

Since it's not trivial to detect which stats are actually "frametime" stats as they can go by different names, the simplest solution is adding a new "unweighted_average" formula. This also allows using it in other cases where it might make more sense, but these should be very rare.

#rnx
#rb andrew.ladenberger
[FYI] ben.woodhouse

[CL 26644283 by tyler staples in ue5-main branch]
2023-07-27 12:35:32 -04:00
ben woodhouse
2ba6a652ed PerfReportTool - Fix a bug with extra links templating. Some file links were not getting templated due to a change in the ExtraLink class detection logic. Make the detection explicit via a 4th optional link template param in the .links file so we can explicitly specify if a link needs template handling by specifying "linkTemplate" or "noLinkTemplate".
If the new param isn't found or "auto" is specified, detection is automatic (based on the presence of http:// or https://)

#rb tyler.staples

[CL 26296250 by ben woodhouse in ue5-main branch]
2023-06-28 15:44:22 -04:00
ben woodhouse
31edc9c43b When sorting columns by diff, keep CSV stats of the same category together. Do an addional stable sort by stat category after the initial sort by diff
[FYI] tyler.staples

[CL 25990671 by ben woodhouse in ue5-main branch]
2023-06-14 15:34:16 -04:00
ben woodhouse
ded3c14346 PerfReportTool 4.105.0: Add support for sorting columns by diff and also filtering out columns based on a diff value threshold.
Adds the following params:
  -sortColumnsByDiff : sorts columns by the max of its diff scores (use with -addDiffRows)
  Note: Diff score corresponds to the value of a column's diff row; the sign is reversed if LowIsBad

  -columnDiffDisplayThreshold <value> : if specified, hides columns with max abs diff value below this threshold

Also: added some helpers to summary tables to find aggregate columns . Added a isCountColumn accessor to avoid string compares

[CL 25982885 by ben woodhouse in ue5-main branch]
2023-06-14 12:34:40 -04:00
ben woodhouse
6785e92518 PerfReportTool - avoid div/0 in autocolorization
[FYI] tyler.staples

[CL 25952964 by ben woodhouse in ue5-main branch]
2023-06-13 11:27:26 -04:00
ben woodhouse
feec4458d9 PerfReportTool 4.104 - Adjust auto-colorization to reduce the color difference when the difference between min/max is very small. Also add a threshold to disable colorization altogether.
[FYI] tyler.staples,tom.holmes

[CL 25950988 by ben woodhouse in ue5-main branch]
2023-06-13 10:20:08 -04:00
ben woodhouse
ea62774e80 CsvConvert 1.05 - automatically repair null metadata if we're adding metadata (this is a common use-case)
CsvStats - fix bug where ReadFileFromLines wouldn't close the file (breaking -inplace mode in CsvConvert)

[CL 25494966 by ben woodhouse in ue5-main branch]
2023-05-16 15:29:42 -04:00
tyler staples
7ab16f4e82 PerfReportTool - Fix divide by 0 issue in MVP calculation.
[FYI] andrew.ladenberger

[CL 25035524 by tyler staples in ue5-main branch]
2023-04-13 20:28:01 -04:00
tyler staples
d2ddfa358a PerfReportTool - Add support for date formats and disabling column word wrap.
#rb ben.woodhouse

[CL 24905945 by tyler staples in ue5-main branch]
2023-04-03 22:30:12 -04:00
andrew ladenberger
b74738167a [PerfReportTool] Removed gamethread histogram from the summary table. This works better as part of the report (see HistogramSummary.cs)
#rb tyler.staples, ben.woodhouse

[CL 24343562 by andrew ladenberger in ue5-main branch]
2023-02-21 13:09:49 -05:00
ben woodhouse
0e7d1d7ea0 PerfReportTool - Add -summaryTableXmlRowSortAppend arg to allow appending values to a summary table's row sort list via the commandline
[CL 24136498 by ben woodhouse in ue5-main branch]
2023-02-11 12:25:29 -05:00
ben woodhouse
f7cd8f51d5 PerfReportTool: add -summaryTableXmlAppend arg for appending stats/metrics to the end of a summary table's filter list via the commandline. This allows us to append multiple stats (-summaryTableXmlSubst only supports one stat per substitution)
[CL 24092968 by ben woodhouse in ue5-main branch]
2023-02-09 00:36:36 -05:00
ben woodhouse
3785b38b51 PerfReportTool - support writing PRC json to separate files, via -summaryTableToJsonSeparateFiles. Also add -summaryTableToJsonNoIndent option
[CL 23462559 by ben woodhouse in ue5-main branch]
2022-12-09 08:21:02 -05:00
ben woodhouse
f02d3c3cc7 PerfReportTool - fix help formatting so it fits in a standard 120-character wide command prompt and make some descriptions more concise.
[FYI] tyler.staples

[CL 23462265 by ben woodhouse in ue5-main branch]
2022-12-09 07:21:26 -05:00
ben woodhouse
a7c96a8ca4 PerfReportTool 4.99.0
- Add support for a default report type. This will be considered if autodetection otherwise fails
- Add a requiredForDefaultFallback param to make specific metadata matches optional if we're falling back. This defaults to 1 (true)
- Add allowAutoDetect flag to reports so they can opt out of autodetection (these need to be specified explicitly)

[CL 23263072 by ben woodhouse in ue5-main branch]
2022-11-24 19:13:30 -05:00
jordan cristiano
4e35c288f8 Added output of per-event mvp, hitches data so we can reference it in summary tables.
#rb tyler.staples
[FYI] andrew.ladenberger, ben.woodhouse

[CL 22999920 by jordan cristiano in ue5-main branch]
2022-11-04 22:29:17 -04:00
tyler staples
c4009bdc7b PerfReportTool - Fix off by one error in regression filter when counting rows in first row group.
+ Also added handling for if all data is part of the same row group or if there are no rows (avoid divide by 0).

#rb ben.woodhouse

[CL 22961417 by tyler staples in ue5-main branch]
2022-11-03 15:02:31 -04:00
tyler staples
3b43312f37 PerfReportTool - Add support for regression and other column filters.
+ Additional column filters can now be created by implementing ISummaryTableColumnFilter and adding it to the list in Program.MakeAdditionalColumnFilters.
+ Moved stat threshold checks into an additional filter.
+ Added a column filter to hide metadata columns.
+ Added a column filter to only show columns where the most recent build has regressed by more than a given threshold.
+ You can now use 'debugShowFilteredColumns' to have columns be grayed out instead of removed. Hovering over them will show the reason they were filtered out (which filter and the reason). This is for debugging purposes and sanity checks to make sure you're not missing important information, especially when iterating on new filters. Some additional settings were added to SummaryTableColumn to support this.
+ Filters can also mark individual cells as 'invalid' to indicate they are not taken into consideration for that filter. These are grayed out with a tooltip when debugShowFilteredColumns is enabled. Currently this does not work in collated views.
+ Added the ability to set tooltips for column headers.
+ Added the ability to add a color modifier for the entire column or individual cells.

#rb ben.woodhouse

[CL 22890943 by tyler staples in ue5-main branch]
2022-11-01 16:30:42 -04:00
tyler staples
b02c452ae8 PerfReportTool - Update budget colour scheme to match auto as it's much easier on the eyes.
#rb ben.woodhouse

[CL 22798511 by tyler staples in ue5-main branch]
2022-10-26 19:10:45 -04:00