Commit Graph

178 Commits

Author SHA1 Message Date
ben woodhouse
52de449da0 PerfReportTool 4.224.1: Derived metadata populates variables
-add dumpVariablesAll param to dump all variables including metadata

[CL 30091707 by ben woodhouse in ue5-main branch]
2023-12-04 14:55:26 -05:00
ben woodhouse
2ea0faefb3 PerfReportTool 4.224.0: Adjust default colorization to make greens and reds lighter and more readable
- Add Variable support for hitch thresholds

[CL 30080549 by ben woodhouse in ue5-main branch]
2023-12-04 10:00:58 -05:00
ben woodhouse
9073333aa2 PerfReportTool 4.223.0. Budget and variable improvements
- Multiplier support for variables. Add multiplier="N" attribute to a variableset or variable
- FPSCharts : Compute frametime color thresholds automatically if MVP thresholds are specified and frametime is not
- Support empty color thresholds (useful if we want to clear via a variable)
- Add support for dumping the variables to the log for each CSV via -dumpVariables param

[CL 30060917 by ben woodhouse in ue5-main branch]
2023-12-01 19:11:26 -05:00
ben woodhouse
b12d868886 PerfReportTool 4.222.0 : support dereferencing variables with array indices for strings separated with commas. Also add validation for variable names.
[CL 29986252 by ben woodhouse in ue5-main branch]
2023-11-29 09:04:49 -05:00
ben woodhouse
1419a64864 Add Contains and Get methods for SummaryTableRowData
[CL 29984224 by ben woodhouse in ue5-main branch]
2023-11-29 06:52:20 -05:00
ben woodhouse
b365599507 PerfReportTool 4.221.0 : Add support for metadata queries and metadata variables (ie {$meta.NAME} ) in derived metadata mappings. This makes these declarations more powerful and reduces duplication
Metadata queries can also be blank if necessary if we want the values to always be set.
SourceName/DestName are still supported for back-compat if no query is provided
Note: There is no globalvariable support here. Solving this is not worth the complexity, performance hit and potential ordering and correctness implications
#tests Passes Test cases with identical results
[FYI] tyler.staples

[CL 29922595 by ben woodhouse in ue5-main branch]
2023-11-24 13:49:43 -05:00
ben woodhouse
72d3559c65 PerfReportTool 4.220: VariableSets support
- Add support for variable sets. This allows us to set blocks of variables based on metadata, which can be accessed via ${NAME} syntax. Variable sets support nesting and can be conditional using metadata queries
  - We support a global variable set (declared under <globalVariableSet> in root), and local variable sets which apply to a given report type only (as <variableSet> children of the ReportType element)

example:
<globalVariableSet>
  <variableSet>
    <var mvpStatName="MVP${meta.targetframerate}"/>
    <var targetFps="${meta.targetframerate}"/>
    <var dynamicResBudget="100"/>
  </variableSet>

  <!-- Switch -->
  <variableSet metadataQuery="platform=Switch">
    <var frameTimeBudget="33.33"/>
    <variableSet metadataQuery="deviceprofile=Switch_Console_Forward*">
      <var dynamicResBudget="92"/>
    </variableSet>
    <variableSet metadataQuery="deviceprofile=Switch_Handheld_Forward*">
      <var dynamicResBudget="88"/>
    </variableSet>
  </variableSet>
</globalVariableSet>

#tests Passes Test cases with identical results (if no changes to the XML)
[FYI] tyler.staples

[CL 29918333 by ben woodhouse in ue5-main branch]
2023-11-24 07:45:23 -05:00
ben woodhouse
17cbcb95f7 Improvements to metadata query evaluation (csvStats lib)
-  Equallity now uses numeric comparisons if appropriate
  - Add a ~= operator for contains
  - Fix a potential infinite loop if a bad term is evaluated

[CL 29918133 by ben woodhouse in ue5-main branch]
2023-11-24 07:29:21 -05:00
ben woodhouse
146dc093bb PerfReportTool 4.210.1 : fix an exception when startTimestamp is not set for some rows
[CL 29915928 by ben woodhouse in ue5-main branch]
2023-11-23 22:04:05 -05:00
ben woodhouse
932a14ce02 PerfReportTool 4.210 - Variables part 1 : Add support for metadata references in ReportTypes XML. CSV Metadata references can now be specified Report xml attributes (and values in select cases) with ${meta.KEY} syntax, e.g offsetY="${meta.minimapoffsety}". Multiple variables can be matched in a single attribute if needed, eg offsetXY="${meta.minimapoffsetx},${meta.minimapoffsety}"
Support is provided for Summary classes via XElement extension helper methods, which now take an optional XmlVariableMappings param which handles the variable resolving. GetAttribute, GetSafeAttribute and GetValue support this.
Non metadata variables are also supported in theory e.g ${MyVariable}, but support for populating these does not exist yet
Various cleanup/refactoring and fixups of old code

#tests Passes Test cases with identical results (if no changes to the XML)
[FYI] tyler.staples

[CL 29912786 by ben woodhouse in ue5-main branch]
2023-11-23 14:06:40 -05:00
tyler staples
a256e0515b PerfReportTool - Fix incorrect base directory
Also removed debug directory handling for launching CSVToSVG in debug mode.

#rnx
#tests prt test cases, prs unit tests, local report generation
#rb ben.woodhouse

[CL 29875869 by tyler staples in ue5-main branch]
2023-11-21 17:34:46 -05:00
ben woodhouse
62e7c2821a PerfReportTool: Remove unnecessary baseOutputDir
[CL 29870089 by ben woodhouse in ue5-main branch]
2023-11-21 13:07:15 -05:00
ben woodhouse
bf257cb06f Upgrade CSV tools to .NET core 6. Includes various required fixes for decompression, json reading, graph component hash generation and exception handling. Performance is better than .Net Framework.
[CL 29839806 by ben woodhouse in ue5-main branch]
2023-11-20 08:04:34 -05:00
ben woodhouse
8d91b3258f PerfReportTool 4.108.0 + CsvToSvg: -maxAutoMaxY support for graphs + other fixes:
- Add maxAutoMaxY param to graphs and csvToSVG params which allows us to clamp the max automatic Y to a particular value. Also:
- Fix some naming clashes when there are mulltiple graphs on a page. This was preventing us from having graphs with different sizes in a perf report
- Add -embedGraphCommandline arg to help debugging
[FYI] tyler.staples

[CL 29745624 by ben woodhouse in ue5-main branch]
2023-11-15 09:55:24 -05:00
ola olsson
36b8421f1f Added the ability to pass in a list of custom legend entries that match the list of csv files & increased the maxy limit to 8M.
#rb ben.woodhouse

[CL 29743496 by ola olsson in ue5-main branch]
2023-11-15 07:46:27 -05:00
ben woodhouse
527126fd2c Rebuild PerfReportTool to fix an exception, and bump version to 4.107.2 (cs change was checked in but exe hadn't been rebuilt)
[CL 29189970 by ben woodhouse in ue5-main branch]
2023-10-27 14:09:15 -04:00
kaleb morris
dec17629ad Fix bug where "minimum" stats that filter out zeros will be zero if their first value is zero
#rb ben.woodhouse, tyler.staples, matt.harris

[CL 29092162 by kaleb morris in ue5-main branch]
2023-10-25 12:21:24 -04:00
ben woodhouse
fcbbb0d785 CsvToSVG 2.58 (and PerfReportTool 4.107.1) : Fix graphing issue with highlightEventRegion where begin can't be null. This allows us to grey out regions at the start of matches
[CL 28945269 by ben woodhouse in ue5-main branch]
2023-10-19 20:14:08 -04:00
tyler staples
342b522b5f PerfReportTool - Fix a crash when reading partial csvs (just the end so we can read metadata).
#rnx
[FYI] ben.woodhouse

[CL 26922403 by tyler staples in ue5-main branch]
2023-08-08 13:28:43 -04:00
tyler staples
48925f6220 PerfReportTool - Added an "unweighted_average" formula to BoundedStatValueSummaries as weighted averages can sometimes be incorrect.
BoundedStatValueSummaries weights the stat averages by the frametimes. Normally it's not an issue to use the same value for both the value and the weight as they'll end up canceling each other out resulting in an unweighted average. However, in the case where some values are much larger than others this can skew the value resulting in a value much different from the weighted average. This is the case for server frametimes as there can be a few very large spikes that end up skewing the weights.

Since it's not trivial to detect which stats are actually "frametime" stats as they can go by different names, the simplest solution is adding a new "unweighted_average" formula. This also allows using it in other cases where it might make more sense, but these should be very rare.

#rnx
#rb andrew.ladenberger
[FYI] ben.woodhouse

[CL 26644283 by tyler staples in ue5-main branch]
2023-07-27 12:35:32 -04:00
ben woodhouse
2ba6a652ed PerfReportTool - Fix a bug with extra links templating. Some file links were not getting templated due to a change in the ExtraLink class detection logic. Make the detection explicit via a 4th optional link template param in the .links file so we can explicitly specify if a link needs template handling by specifying "linkTemplate" or "noLinkTemplate".
If the new param isn't found or "auto" is specified, detection is automatic (based on the presence of http:// or https://)

#rb tyler.staples

[CL 26296250 by ben woodhouse in ue5-main branch]
2023-06-28 15:44:22 -04:00
ben woodhouse
31edc9c43b When sorting columns by diff, keep CSV stats of the same category together. Do an addional stable sort by stat category after the initial sort by diff
[FYI] tyler.staples

[CL 25990671 by ben woodhouse in ue5-main branch]
2023-06-14 15:34:16 -04:00
ben woodhouse
ded3c14346 PerfReportTool 4.105.0: Add support for sorting columns by diff and also filtering out columns based on a diff value threshold.
Adds the following params:
  -sortColumnsByDiff : sorts columns by the max of its diff scores (use with -addDiffRows)
  Note: Diff score corresponds to the value of a column's diff row; the sign is reversed if LowIsBad

  -columnDiffDisplayThreshold <value> : if specified, hides columns with max abs diff value below this threshold

Also: added some helpers to summary tables to find aggregate columns . Added a isCountColumn accessor to avoid string compares

[CL 25982885 by ben woodhouse in ue5-main branch]
2023-06-14 12:34:40 -04:00
ben woodhouse
6785e92518 PerfReportTool - avoid div/0 in autocolorization
[FYI] tyler.staples

[CL 25952964 by ben woodhouse in ue5-main branch]
2023-06-13 11:27:26 -04:00
ben woodhouse
feec4458d9 PerfReportTool 4.104 - Adjust auto-colorization to reduce the color difference when the difference between min/max is very small. Also add a threshold to disable colorization altogether.
[FYI] tyler.staples,tom.holmes

[CL 25950988 by ben woodhouse in ue5-main branch]
2023-06-13 10:20:08 -04:00