Commit Graph

212 Commits

Author SHA1 Message Date
ben woodhouse
3cbdfb4570 CsvToSvg Fix issue when minX/MaxX is specified and filteredSampleCount is incorrectly calculated
[CL 34454654 by ben woodhouse in ue5-main branch]
2024-06-18 07:33:34 -04:00
ben woodhouse
8f18ac41bd CsvToSvg 3.62 - Improve visual clutter due to too many events:
- Merge displayed event names if they're too close together, respecting priority (priority is implicit based on param order)
- Display nearby event names in an interactive panel alongside stats
- Truncate displayed events on the graph to 32 characters
Note: you can pass -showAllEvents to disable the new merging/truncation behaviour

[CL 34447676 by ben woodhouse in ue5-main branch]
2024-06-17 20:01:58 -04:00
ben woodhouse
cebfe25fa1 PRT (minor) Fix mapOverlaySummary logging
[CL 34446622 by ben woodhouse in ue5-main branch]
2024-06-17 19:41:57 -04:00
tyler staples
f4a061437b PRT - Switch error log to stderr
Missed this one in the original pass.

[FYI] ben.woodhouse
#rnx

[CL 34386520 by tyler staples in ue5-main branch]
2024-06-14 18:03:32 -04:00
imaad farooqui
5b7549c087 Extend RegressionReport tool to have an extra flag which dumps a content summary to a JSON file
Create a slack bot which reads this JSON content and sends an alert in a specific channel to update users if there was were any regressions
#rb jeremie.roy
#jira UE-216622

[CL 34160685 by imaad farooqui in ue5-main branch]
2024-06-06 12:09:15 -04:00
ben woodhouse
a4f941b75f PerfReportTool 4.237.3: Variable Resolution and graceful failure improvements/bugfixes
- Fix a bug in variable resolution for array variables where the whole field would be replaced instead of just the variable
- Replace variables which fail to resolve with an empty string instead of leaving the variable name in place. This is easier for calling code to detect and handle
- Add a graceful failures in a few places to allow us to generate a report if variables fail to resolve

[CL 34155759 by ben woodhouse in ue5-main branch]
2024-06-06 09:53:59 -04:00
imaad farooqui
a9117aec46 Transforms a CSV report and threshold data to generate a HTML report which highlights if any regressions occurred at a high level
#jira UE-215585

[CL 34099224 by imaad farooqui in ue5-main branch]
2024-06-04 13:30:52 -04:00
ben woodhouse
e7586403de Make padding trailing digits optional using the -sortTrailingDigitsAsNumeric param since it can cause issues
[FYI] tom.holmes

[CL 34062555 by ben woodhouse in ue5-main branch]
2024-06-03 07:29:14 -04:00
tyler staples
2d92f41356 PRT - Write error logs to stderr instead of stdout.
#rnx

[CL 33986372 by tyler staples in ue5-main branch]
2024-05-29 16:50:21 -04:00
ben woodhouse
143b9b0331 PerfReportTool 4.237.0 - Make summary table sorting smarter. If a column with string values includes an integer suffix, left pad with zeroes for sorting purposes
[CL 33974852 by ben woodhouse in ue5-main branch]
2024-05-29 11:20:53 -04:00
tyler staples
d3cfd48be4 PRT - Guard against divide by 0 in BoundedStatValuesSummary
#rnx

#rb ben.woodhouse, kaleb.morris

[CL 33675340 by tyler staples in ue5-main branch]
2024-05-15 19:51:49 -04:00
tyler staples
c0dee276eb PRT - Fix null object exception in bounded summary stat table.
#rnx
#rb matt.harris

[CL 33664264 by tyler staples in ue5-main branch]
2024-05-15 13:55:32 -04:00
matt harris
65b80a11bb PerfReportTool v4.236.0 - Added Wildcard Stat support to BoundedStatSummary. This allows ReportTypes.xml to specify '*' for generated columns in a summary table. The tool will now attempt to match that wildcard to all possible stats, generating new columns for each. Wildcard symbol needs to be provided in the column name, summaryState name and stat name, or it will skip that stat name. Supports otherStat as well so can be used with the Ratio formula.
#rb ben.woodhouse, kaleb.morris
#tests Built in Release and tested both standalone and via PRS
[RN] PerfReportTool v4.236.0 Added Wildcard Stat support to BoundedStatSummary. This allows ReportTypes.xml to specify '*' for generated columns in a summary table
[FYI] richard.smith

[CL 33652701 by matt harris in ue5-main branch]
2024-05-15 04:30:44 -04:00
ben woodhouse
0d68778837 CsvCollate 1.35 - add StartEvent param so we can trim the start of the CSVs we're combining. This is useful for syncing up timelines
[CL 32543012 by ben woodhouse in ue5-main branch]
2024-03-27 10:22:52 -04:00
ben woodhouse
cb37d31a43 PerfReportTool 4.235.0. Add hideInDetailedReports attribute to summaries allowing them to be suppressed. This is useful if you only want to emit summary table row data for a given summary
[CL 32185090 by ben woodhouse in ue5-main branch]
2024-03-12 13:33:03 -04:00
ben woodhouse
f27a3c8875 PerfReportTool 4.234.0
- Add support for startEvent and stopEvent in graphs
- Add requiredEvents field for graphs so we can suppress graphs based on whether particular events are present

[CL 32016929 by ben woodhouse in ue5-main branch]
2024-03-05 08:03:21 -05:00
ben woodhouse
3c075bb5c1 CSVToSVG 3.61
- Add -startEvent and -endEvent params to start and stop graphing at a particular event
- The params -startEventOffset and -endEventOffset allow you to offset the start/end by a particular number of frames
- If multiple CSVs are being processed, the data is cropped so frame counts start from zero
- The existing -maxx and -maxy parameters are relative to the adjusted CSV (e.g so -maxx=200 with -startEvent gc will capture 200 frames from the first gc)
- Fix bug with legend sorting. Frame is now always top
- Fix bug with auto Y axis scaling when a range is specified. Use the range instead of the whole CSV

Common
- Fix commandline parsing for negative numbers. Negative numbers are always recognized as values rather than param names

[FYI] tyler.staples

[CL 32016893 by ben woodhouse in ue5-main branch]
2024-03-05 08:00:20 -05:00
ben woodhouse
d4d2df2b83 PerfReportTool 4.233.0 : Unify wildcard handling for summary tables so it correctly matches mid-string wildcards. Also optimize using a binary search
[CL 31979848 by ben woodhouse in ue5-main branch]
2024-03-03 10:27:37 -05:00
ben woodhouse
1f59f63a96 PerfReportTool 4.232.0: Add support for stat type qualifiers in summary tables so we can identify stats by their type to avoid ambiguity, e.g [meta]deviceprofile. Prefixes are :
[metric] SummaryTableMetric
[csv] CsvStatAverage
[meta] CsvMetadata
[toolmeta] ToolMetadata

[CL 31978791 by ben woodhouse in ue5-main branch]
2024-03-02 20:24:47 -05:00
ben woodhouse
90a828c869 PerfReportTool: Also output the elementType in the header tooltip (E.g CsvMetadata, SummaryTableMetric etc)
[CL 31599556 by ben woodhouse in ue5-main branch]
2024-02-17 07:08:25 -05:00
ben woodhouse
b8f8c1b7b2 PerfReportTool: When stat names are remapped, display the original name as a tooltip
[CL 31578316 by ben woodhouse in ue5-main branch]
2024-02-16 15:42:57 -05:00
ben woodhouse
6b06bd7eaa PerfReportTool 4.230.0 - Add support for variables which can be used outside of individual CSV reporting. Any variable declared in a variableSet without a metadata query can be used in this way. This is useful if we want to access variables in SummaryTable definitions for example. Also strip whitespace in summary table row sort and filter lists
[CL 30955693 by ben woodhouse in ue5-main branch]
2024-01-27 11:28:43 -05:00
bruno ferreira1
963571427f Adding -showmin and -showmax to csvinfo
[REVIEW] [at]tyler.staples
#rnx

[CL 30930359 by bruno ferreira1 in ue5-main branch]
2024-01-26 12:56:37 -05:00
tyler staples
ed483b845c PRT - Update help text to include new dynamic graph args.
Also removed deprecated externalGraphs arg

#rnx

#rb ben.woodhouse

[CL 30719102 by tyler staples in ue5-main branch]
2024-01-19 12:23:26 -05:00
ben woodhouse
a941e9c6bf PerfReportTool: Minor tweaks to heading formatting for better use of vertical space. Graphs now use h3 instead of h2
[CL 30718644 by ben woodhouse in ue5-main branch]
2024-01-19 12:12:16 -05:00