Commit Graph

219 Commits

Author SHA1 Message Date
Ryan Hummer
4af2fd066d Updating Dev-Release-5.5 from Main at CL #36144969
#okforversepublic

[CL 36146571 by Ryan Hummer in Dev-5.5 branch]
2024-09-10 10:26:02 -04:00
brendan lienau
14c67329dd Updates CSVTools projects and binaries to target anycpu instead of only x86_64
[FYI] Tyler.Staples, Ben.Woodhouse
#tests FC diffs on before and after results of PerfReportFort test cases

[CL 35348616 by brendan lienau in ue5-main branch]
2024-08-06 14:57:13 -04:00
tom holmes
cba6447212 Enhanced access to variables in PRT without CSVs
"-metadataProxy name" looks up named metadata proxy in ReportTypes.xml csvMetadataProxies
  "-overrideMetadata name=value,name=value,name=value" supplies explicit metadata pairs from the commandline
Added query param to PRS to display variables based upon a CSV in the server database
  "/variables?id=csvid[&all=true|false]"
Updated binaries

#rb ben.woodhouse, tyler.staples
#tests PRT test cases, local perf server

[CL 35279307 by tom holmes in ue5-main branch]
2024-08-02 13:00:52 -04:00
tom holmes
347ca2cf1d Add ability for PRT to dump variables to json: -dumpVariablesToJson and -dumpAllVariablesToJson
Adds -noReports to skip report writing
Rebuild relevent binary and pdb

#rb tyler.staples, ben.woodhouse
#rnx
#tests local PRT testing

[CL 35106556 by tom holmes in ue5-main branch]
2024-07-26 08:30:52 -04:00
tyler staples
b1969b8329 PRT - Add sort option to bucket summary stats. Also sort event summaries by count.
+ Add the option to sort bucket summary tables by a particular bucket index (asc/desc) or by the stat name.
+ Add the option to specificy the bucket precision in the header column.
+ Sort event summaries by count by default so most frequent events are at the top.

Binary changes will be included before submission.

#rnx
#rb ben.woodhouse

[CL 35021236 by tyler staples in ue5-main branch]
2024-07-23 11:38:44 -04:00
tyler staples
eeff5dcd62 PRT - Handle missing graphs gracefully in bulk mode and log an error instead of exiting.
This is a common mistake when editing the XML so we don't want it to prevent generating future reports.

#rnx
[FYI] ben.woodhouse

[CL 34920797 by tyler staples in ue5-main branch]
2024-07-18 19:09:29 -04:00
ben woodhouse
bf86c1d0a3 CsvToSvg 3.64: Fix javascript variable clash with event display
#rb tyler.staples

[CL 34742704 by ben woodhouse in ue5-main branch]
2024-06-28 12:51:09 -04:00
ben woodhouse
3cbdfb4570 CsvToSvg Fix issue when minX/MaxX is specified and filteredSampleCount is incorrectly calculated
[CL 34454654 by ben woodhouse in ue5-main branch]
2024-06-18 07:33:34 -04:00
ben woodhouse
8f18ac41bd CsvToSvg 3.62 - Improve visual clutter due to too many events:
- Merge displayed event names if they're too close together, respecting priority (priority is implicit based on param order)
- Display nearby event names in an interactive panel alongside stats
- Truncate displayed events on the graph to 32 characters
Note: you can pass -showAllEvents to disable the new merging/truncation behaviour

[CL 34447676 by ben woodhouse in ue5-main branch]
2024-06-17 20:01:58 -04:00
ben woodhouse
cebfe25fa1 PRT (minor) Fix mapOverlaySummary logging
[CL 34446622 by ben woodhouse in ue5-main branch]
2024-06-17 19:41:57 -04:00
tyler staples
f4a061437b PRT - Switch error log to stderr
Missed this one in the original pass.

[FYI] ben.woodhouse
#rnx

[CL 34386520 by tyler staples in ue5-main branch]
2024-06-14 18:03:32 -04:00
imaad farooqui
5b7549c087 Extend RegressionReport tool to have an extra flag which dumps a content summary to a JSON file
Create a slack bot which reads this JSON content and sends an alert in a specific channel to update users if there was were any regressions
#rb jeremie.roy
#jira UE-216622

[CL 34160685 by imaad farooqui in ue5-main branch]
2024-06-06 12:09:15 -04:00
ben woodhouse
a4f941b75f PerfReportTool 4.237.3: Variable Resolution and graceful failure improvements/bugfixes
- Fix a bug in variable resolution for array variables where the whole field would be replaced instead of just the variable
- Replace variables which fail to resolve with an empty string instead of leaving the variable name in place. This is easier for calling code to detect and handle
- Add a graceful failures in a few places to allow us to generate a report if variables fail to resolve

[CL 34155759 by ben woodhouse in ue5-main branch]
2024-06-06 09:53:59 -04:00
imaad farooqui
a9117aec46 Transforms a CSV report and threshold data to generate a HTML report which highlights if any regressions occurred at a high level
#jira UE-215585

[CL 34099224 by imaad farooqui in ue5-main branch]
2024-06-04 13:30:52 -04:00
ben woodhouse
e7586403de Make padding trailing digits optional using the -sortTrailingDigitsAsNumeric param since it can cause issues
[FYI] tom.holmes

[CL 34062555 by ben woodhouse in ue5-main branch]
2024-06-03 07:29:14 -04:00
tyler staples
2d92f41356 PRT - Write error logs to stderr instead of stdout.
#rnx

[CL 33986372 by tyler staples in ue5-main branch]
2024-05-29 16:50:21 -04:00
ben woodhouse
143b9b0331 PerfReportTool 4.237.0 - Make summary table sorting smarter. If a column with string values includes an integer suffix, left pad with zeroes for sorting purposes
[CL 33974852 by ben woodhouse in ue5-main branch]
2024-05-29 11:20:53 -04:00
tyler staples
d3cfd48be4 PRT - Guard against divide by 0 in BoundedStatValuesSummary
#rnx

#rb ben.woodhouse, kaleb.morris

[CL 33675340 by tyler staples in ue5-main branch]
2024-05-15 19:51:49 -04:00
tyler staples
c0dee276eb PRT - Fix null object exception in bounded summary stat table.
#rnx
#rb matt.harris

[CL 33664264 by tyler staples in ue5-main branch]
2024-05-15 13:55:32 -04:00
matt harris
65b80a11bb PerfReportTool v4.236.0 - Added Wildcard Stat support to BoundedStatSummary. This allows ReportTypes.xml to specify '*' for generated columns in a summary table. The tool will now attempt to match that wildcard to all possible stats, generating new columns for each. Wildcard symbol needs to be provided in the column name, summaryState name and stat name, or it will skip that stat name. Supports otherStat as well so can be used with the Ratio formula.
#rb ben.woodhouse, kaleb.morris
#tests Built in Release and tested both standalone and via PRS
[RN] PerfReportTool v4.236.0 Added Wildcard Stat support to BoundedStatSummary. This allows ReportTypes.xml to specify '*' for generated columns in a summary table
[FYI] richard.smith

[CL 33652701 by matt harris in ue5-main branch]
2024-05-15 04:30:44 -04:00
ben woodhouse
0d68778837 CsvCollate 1.35 - add StartEvent param so we can trim the start of the CSVs we're combining. This is useful for syncing up timelines
[CL 32543012 by ben woodhouse in ue5-main branch]
2024-03-27 10:22:52 -04:00
ben woodhouse
cb37d31a43 PerfReportTool 4.235.0. Add hideInDetailedReports attribute to summaries allowing them to be suppressed. This is useful if you only want to emit summary table row data for a given summary
[CL 32185090 by ben woodhouse in ue5-main branch]
2024-03-12 13:33:03 -04:00
ben woodhouse
f27a3c8875 PerfReportTool 4.234.0
- Add support for startEvent and stopEvent in graphs
- Add requiredEvents field for graphs so we can suppress graphs based on whether particular events are present

[CL 32016929 by ben woodhouse in ue5-main branch]
2024-03-05 08:03:21 -05:00
ben woodhouse
3c075bb5c1 CSVToSVG 3.61
- Add -startEvent and -endEvent params to start and stop graphing at a particular event
- The params -startEventOffset and -endEventOffset allow you to offset the start/end by a particular number of frames
- If multiple CSVs are being processed, the data is cropped so frame counts start from zero
- The existing -maxx and -maxy parameters are relative to the adjusted CSV (e.g so -maxx=200 with -startEvent gc will capture 200 frames from the first gc)
- Fix bug with legend sorting. Frame is now always top
- Fix bug with auto Y axis scaling when a range is specified. Use the range instead of the whole CSV

Common
- Fix commandline parsing for negative numbers. Negative numbers are always recognized as values rather than param names

[FYI] tyler.staples

[CL 32016893 by ben woodhouse in ue5-main branch]
2024-03-05 08:00:20 -05:00
ben woodhouse
d4d2df2b83 PerfReportTool 4.233.0 : Unify wildcard handling for summary tables so it correctly matches mid-string wildcards. Also optimize using a binary search
[CL 31979848 by ben woodhouse in ue5-main branch]
2024-03-03 10:27:37 -05:00