Commit Graph

104 Commits

Author SHA1 Message Date
ben woodhouse
e34e6acc24 CsvToSVG 2.55 - fix bug where colors are all the same with multiple CSVs
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19966504 via CL 19966958 via CL 19967479
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 19971908 by ben woodhouse in ue5-main branch]
2022-04-28 18:56:26 -04:00
ben woodhouse
780fc256db CsvToSvg 2.54 - fix bug with graph smoothing when kernel size is small: Clamp the downsample level so it can't go negative
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19939969 via CL 19939983 via CL 19940000
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 19945635 by ben woodhouse in ue5-main branch]
2022-04-27 16:06:34 -04:00
ben woodhouse
b844536232 CsvInfo 1.05 - don't read the full CSV unless it's necessary. This is roughly 100x times faster for binary CSVs, and 2x as fast for text
- add -forceFullRead option to force a full read (useful for validation)

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19900506 via CL 19900521 via CL 19900526
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v940-19807014)

[CL 19902188 by ben woodhouse in ue5-main branch]
2022-04-25 10:23:35 -04:00
ben woodhouse
426637ef26 CsvInfo 1.04 - add -showTotals option.
- display sample count and output it to json
- add proper support for totals and averages in json. When these are requested, we output stats as a dict rather than an array

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19896982 via CL 19896983 via CL 19896984
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v940-19807014)

[CL 19897246 by ben woodhouse in ue5-main branch]
2022-04-24 13:09:58 -04:00
ben woodhouse
45fdd234a9 PerfReportTool 4.75 - Add logging around map image handling
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19862730 via CL 19866748 via CL 19866963
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v940-19807014)

[CL 19871403 by ben woodhouse in ue5-main branch]
2022-04-22 16:06:16 -04:00
ben woodhouse
cf8c132bde PerfReportTool 4.74
- Link templates are only added for extra relative links. Links starting with http:// or https:// are not replaced

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19828731 via CL 19828779 via CL 19828794
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v940-19807014)

[CL 19835207 by ben woodhouse in ue5-main branch]
2022-04-20 14:43:25 -04:00
ben woodhouse
85f75b9bd0 CsvToSVG 2.53 - Refactor and optimize SmoothStats via add a SmoothKernel class.
- Add support for downsampled smoothing for an up to 16x speedup
 - More efficient threading
 - Minor perflog fixes
 - Add -PerfLog arg for detailed timings

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19786432 via CL 19786566 via CL 19786610
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803461 by ben woodhouse in ue5-main branch]
2022-04-19 00:20:54 -04:00
ben woodhouse
e0529f7769 CsvToSvg 2.52
- Optimize interactive graphs, removing redundant characters from number strings. Saves about 1MB in a typical 20MB report

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19782426 via CL 19782438 via CL 19782441
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803418 by ben woodhouse in ue5-main branch]
2022-04-19 00:19:51 -04:00
ben woodhouse
2709ab9660 Shrink map summary images to the destination size rather than just copying
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19782302 via CL 19782307 via CL 19782309
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803417 by ben woodhouse in ue5-main branch]
2022-04-19 00:19:45 -04:00
ben woodhouse
6c40ffa0aa Add support for fixed point graphs (enabled by default)
Fixed point graphs are smaller because they don't include the decimal point. They're also faster to write. We need to apply a scale in order to have subpixel accuracy. The scale is specified by -fixedpointPrecisionScale <N> (default is 2)
- This improves perf report generation speed by ~15% and report size by 20%.

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19782268 via CL 19782269 via CL 19782270
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803416 by ben woodhouse in ue5-main branch]
2022-04-19 00:19:38 -04:00
ben woodhouse
5229ec9b03 PerfReportTool 4.72 - use CsvToSvgLib async tasks instead of processes by default (override to legacy behavior with -csvToSvgProcesses). This reduced bulk report generation from 12s to 7.7s in tests
- fix a bug in custom commandline parsing where last character would be skipped
- Update binaries for PerfReportTool, CsvToSvg, CsvToSvgLib, CsvStats

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19781993 via CL 19781999 via CL 19782001
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803413 by ben woodhouse in ue5-main branch]
2022-04-19 00:19:31 -04:00
ben woodhouse
da0f70912a CsvToSvg 2.50. Refactor code to move graph generation into a separate lib, so we can use it from PerfReportTool directly instead of executing a separate process and reading all the Csv data multiple times. This also cleans up the code in CsvToSvg considerably.
- Graph Generation is now handled by a GraphGenerator class with MakeGraphs async and synchronous methods. Settings are passed via a GraphParams struct.
- Optimize various functions to speed up graph generation.
Generating graphs for a report is now around 25% faster.

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19781991 via CL 19781996 via CL 19782000
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803410 by ben woodhouse in ue5-main branch]
2022-04-19 00:19:24 -04:00
ben woodhouse
ffa293def7 CsvConvert - Fix exception which occurs if setMetadataFile isn't passed on the commandline
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19725891 via CL 19726236 via CL 19726839
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19728300 by ben woodhouse in ue5-main branch]
2022-04-12 16:31:33 -04:00
ben woodhouse
f5afb924b9 CsvInfo 1.03: -quiet mode. Useful if you only care about the return code (ie to determine if a CSV is valid)
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19686927 via CL 19686937 via CL 19686945
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19687531 by ben woodhouse in ue5-main branch]
2022-04-08 11:08:22 -04:00
ben woodhouse
ac128d84de CsvConvert 1.03
- Add setMetadataFile arg for setting/overriding metadata. This is more robust than passing it on the commandline. Input is a text file where each line corresponds to a metadata key/value and each line has the format <Key>=<Value>

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19686921 via CL 19686936 via CL 19686941
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19687528 by ben woodhouse in ue5-main branch]
2022-04-08 11:08:14 -04:00
ben woodhouse
8f29d53bc6 PerfReportTool updates to support new PerfReportServer
PerfReportTool 4.71
 - Add support for summary table link templates that can be replaced later (e.g by a web server). E.g {LinkTemplate:Csv:<CSVID>}. Pass -linkTemplates to enable. This works for Csv and Report links, as well as ExtraLinks
 - Add -csvList and -PRCList args for specifying a list of PRCs or CSVs
 - Add -summaryTableXML arg for specifying an additional XML file containing summary tables
 - Summary table titles can now be specified with -summaryTitle instead of this being automatic

All CSV tools:
 - Add response file commandline support via -response=<filename>


#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19686909 via CL 19686925 via CL 19686934
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19687527 by ben woodhouse in ue5-main branch]
2022-04-08 11:08:05 -04:00
ben woodhouse
9f3e484f44 CsvInfo - Fix stat array json output
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19419881 via CL 19421464 via CL 19429070 via CL 19429184
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)

[CL 19432570 by ben woodhouse in ue5-main branch]
2022-03-18 02:33:28 -04:00
ben woodhouse
6f8224a2d0 CsvInfo 1.02 - json support. Pass -toJson <filename> to output Csv metadata and stats to a json file.
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19419507 via CL 19421375 via CL 19429041 via CL 19429172
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)

[CL 19432569 by ben woodhouse in ue5-main branch]
2022-03-18 02:33:21 -04:00
ben woodhouse
7dade6bbe5 Fix bug with stat wildcard matching which caused stats to report a match if a wildcard was present even if the start or end of the stat name didn't match
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19112156 via CL 19112168 via CL 19112169 via CL 19112271
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)

[CL 19152812 by ben woodhouse in ue5-main branch]
2022-02-25 14:16:07 -05:00
ben woodhouse
1124980b31 PerfReportTool 4.70. Make adding duplicate metadata keys non-fatal. A warning will be emitted to the log instead.
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 18826796 via CL 18826862 via CL 18826914 via CL 18835867 via CL 18836719
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v910-18824042)

[CL 18836909 by ben woodhouse in ue5-main branch]
2022-02-02 18:42:20 -05:00
ben woodhouse
0654534f3c PerfReportTool 4.69: Add -maxFileAgeDays <N> arg - CSV or PRC files older than this will be ignored. This is faster than querying timestamp metadata, especially when reading from network drives.
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 18554671 via CL 18554672 via CL 18554673 via CL 18554686 via CL 18554690
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18554691 by ben woodhouse in ue5-release-engine-test branch]
2022-01-08 13:13:03 -05:00
robert millar
5f62e8e592 Expand support for wildcard to support wildcard at start, end or both of filter string.
Fix error message on missing metadata.

#rb ben.woodhouse

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 18412962 via CL 18413177 via CL 18413235 via CL 18434949 via CL 18437302
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18437359 by robert millar in ue5-release-engine-test branch]
2021-12-10 20:37:20 -05:00
ben woodhouse
cce1015f02 PerfReportTool 4.68 - Fix peak summary not emitting SummaryTableData when detailed reports are disabled
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 18344863 via CL 18344911 via CL 18344933 via CL 18348483 via CL 18348520
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18348589 by ben woodhouse in ue5-release-engine-test branch]
2021-12-01 21:14:47 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00