Commit Graph

26 Commits

Author SHA1 Message Date
ben woodhouse
b844536232 CsvInfo 1.05 - don't read the full CSV unless it's necessary. This is roughly 100x times faster for binary CSVs, and 2x as fast for text
- add -forceFullRead option to force a full read (useful for validation)

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19900506 via CL 19900521 via CL 19900526
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v940-19807014)

[CL 19902188 by ben woodhouse in ue5-main branch]
2022-04-25 10:23:35 -04:00
ben woodhouse
85f75b9bd0 CsvToSVG 2.53 - Refactor and optimize SmoothStats via add a SmoothKernel class.
- Add support for downsampled smoothing for an up to 16x speedup
 - More efficient threading
 - Minor perflog fixes
 - Add -PerfLog arg for detailed timings

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19786432 via CL 19786566 via CL 19786610
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803461 by ben woodhouse in ue5-main branch]
2022-04-19 00:20:54 -04:00
ben woodhouse
5229ec9b03 PerfReportTool 4.72 - use CsvToSvgLib async tasks instead of processes by default (override to legacy behavior with -csvToSvgProcesses). This reduced bulk report generation from 12s to 7.7s in tests
- fix a bug in custom commandline parsing where last character would be skipped
- Update binaries for PerfReportTool, CsvToSvg, CsvToSvgLib, CsvStats

#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19781993 via CL 19781999 via CL 19782001
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v939-19570697)

[CL 19803413 by ben woodhouse in ue5-main branch]
2022-04-19 00:19:31 -04:00
ben woodhouse
8f29d53bc6 PerfReportTool updates to support new PerfReportServer
PerfReportTool 4.71
 - Add support for summary table link templates that can be replaced later (e.g by a web server). E.g {LinkTemplate:Csv:<CSVID>}. Pass -linkTemplates to enable. This works for Csv and Report links, as well as ExtraLinks
 - Add -csvList and -PRCList args for specifying a list of PRCs or CSVs
 - Add -summaryTableXML arg for specifying an additional XML file containing summary tables
 - Summary table titles can now be specified with -summaryTitle instead of this being automatic

All CSV tools:
 - Add response file commandline support via -response=<filename>


#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19686909 via CL 19686925 via CL 19686934
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19687527 by ben woodhouse in ue5-main branch]
2022-04-08 11:08:05 -04:00
ben woodhouse
7dade6bbe5 Fix bug with stat wildcard matching which caused stats to report a match if a wildcard was present even if the start or end of the stat name didn't match
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 19112156 via CL 19112168 via CL 19112169 via CL 19112271
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)

[CL 19152812 by ben woodhouse in ue5-main branch]
2022-02-25 14:16:07 -05:00
ben woodhouse
4df9aadc8c CsvConvert 1.02
-Add input format detection for csvConvert. If not specified, input format and compression will be used for output.
-Add -setMetadata key=value;... arg for overwriting metadata
-(CsvStats) make sure commandline metadata is written at the end when writing out in csv format

#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 17625367 via CL 17625371 via CL 17625373 via CL 17625375 via CL 17625378 via CL 17625698
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v871-17566257)

[CL 17625701 by ben woodhouse in ue5-main branch]
2021-09-25 20:56:41 -04:00
aurel cordonnier
e3f7878676 Merge from Release-Engine-Test @ 17462327 to UE5/Main
This represents UE4/Main @17430120 and Dev-PerfTest @17437669

[CL 17463546 by aurel cordonnier in ue5-main branch]
2021-09-08 16:42:26 -04:00
ben woodhouse
e3ad811d6b Add missing file from CL 16110876
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16110880 via CL 16110883 via CL 16110885 via CL 16110886
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)

[CL 16117146 by ben woodhouse in ue5-main branch]
2021-04-26 14:45:34 -04:00
ben woodhouse
08525cb9cf PerfReportTool 4.33
Open metadata and .bin files readonly

#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16013849 via CL 16013851 via CL 16013859 via CL 16043212
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)

[CL 16116794 by ben woodhouse in ue5-main branch]
2021-04-26 14:22:36 -04:00
Marc Audy
01b7c9f4f5 Merge UE5/RES @ 15958325 to UE5/Main
This represents UE4/Main @ 15913390 and Dev-PerfTest @ 15913304

[CL 15958515 by Marc Audy in ue5-main branch]
2021-04-08 14:32:07 -04:00
Marc Audy
0cbbc781ca Merge UE5/Release-Engine-Staging @ 15740152 to UE5/Main
This represents UE4/Main @ 15709114

[CL 15740605 by Marc Audy in ue5-main branch]
2021-03-18 15:20:03 -04:00
Marc Audy
ada7c144fa Merge //UE5/Release-Engine-Staging @14903491 to //UE5/Main
[CL 14906022 by Marc Audy in ue5-main branch]
2020-12-11 14:21:20 -04:00
Marc Audy
7379fa99c5 Merging //UE5/Release-Engine-Staging to Main (//UE5/Main) @ 14229157
[CL 14233282 by Marc Audy in ue5-main branch]
2020-09-01 14:07:48 -04:00
Marc Audy
a7c9001a94 Merging //UE5/Release-Engine-Staging to Main (//UE5/Main) @ 14075166
#rb
#rnx

[CL 14075271 by Marc Audy in ue5-main branch]
2020-08-11 01:36:57 -04:00
Marc Audy
11f5b21210 Merging //UE5/Release-Engine-Staging @ 13752110 to Main (//UE5/Main)
#rnx

[CL 13753156 by Marc Audy in ue5-main branch]
2020-06-23 18:40:00 -04:00
Ryan Durand
9ef3748747 Updating copyrights for Engine Programs.
#rnx
#rb none
#jira none

#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869242 in //Fortnite/Release-12.00/... via CL 10869536
#ROBOMERGE-BOT: FORTNITE (Main -> Dev-EngineMerge) (v613-10869866)

[CL 10870955 by Ryan Durand in Main branch]
2019-12-26 23:01:54 -05:00
ben woodhouse
7b101a6080 CsvCollate 1.32- metadata filtering support:
-metadatafilter "key0=value0,key1=value1,..."

[FYI] serge.bernier,mickael.gilabert


#ROBOMERGE-SOURCE: CL 10761485 via CL 10761498
#ROBOMERGE-BOT: (v610-10636431)

[CL 10761500 by ben woodhouse in Main branch]
2019-12-17 09:18:04 -05:00
ben woodhouse
c8e0802da7 CsvToSVG 2.3
- Batched + multithreaded support via response files

PerfReportTool 4.00
 - Support batched graph generation. Increases perf by 30% over old multi process method. Significantly lower CPU and disk usage too. Enable with -batchedgraphs
 - Fixed determinism issues with report generation. Test cases are now identical between runs



#ROBOMERGE-SOURCE: CL 10318336 via CL 10318337
#ROBOMERGE-BOT: (v593-10286020)

[CL 10318338 by ben woodhouse in Main branch]
2019-11-20 07:46:12 -05:00
ben woodhouse
260464e45c PerfReportTool 3.92
Map overlay improvements:
- format improvements
- line color event support
- start event
- line strip resets based on distance
- support for unstripped CSV for certain summaries (e.g mapoverlay and file IO), configurable via XML
- stripped stats with total of zero (workaround for texture streaming stats getting set with zeros in shipping)


#ROBOMERGE-SOURCE: CL 10283452 via CL 10283453
#ROBOMERGE-BOT: (v591-10236483)

[CL 10283454 by ben woodhouse in Main branch]
2019-11-17 21:03:32 -05:00
ben woodhouse
ea6398f040 CsvCollate update - Process one CSV at a time when averaging (to avoid O(N^2) memory use, and output a stat reporting number of CSVs per frame.
#ROBOMERGE-SOURCE: CL 8176983 via CL 8197295
#ROBOMERGE-BOT: (v401-8057353)

[CL 8197495 by ben woodhouse in Main branch]
2019-08-21 17:23:47 -04:00
ben woodhouse
234b307d09 CsvCollate 1.25
Averaging no longers shortens the output CSV to the minimum CSV length. Instead, stat frame values are averaged across the CSVs we have frames for. This allows for meaningful comparisons of large sets of CSVs
TODO: process the CSVs one at a time to reduce peak memory use


#ROBOMERGE-SOURCE: CL 8069626 via CL 8095248
#ROBOMERGE-BOT: (v401-8057353)

[CL 8095363 by ben woodhouse in Main branch]
2019-08-16 13:36:20 -04:00
ben woodhouse
937418f897 PerfReportTool 3.85:
Bulk query improvements:
- Add -beginEvent and -endEvent arguments to generate perf reports only within particular events (e.g skydiving begin/skydiving end)
- Add -summaryTableFilename arg so we can override the summary table filename
[FYI] andrew.ladenberger


#ROBOMERGE-SOURCE: CL 7984408 via CL 7996448
#ROBOMERGE-BOT: (v396-7974030)

[CL 7996657 by ben woodhouse in Main branch]
2019-08-13 14:21:55 -04:00
ben woodhouse
0f76c47673 PerfReportTool 3.84. LLM improvements
- Fix an exception when a CSV has no metadata (e.g LLM)
- Improve peak summary. Add support for the <stats> element (no longer need to rely on stats being referenced in graphs)
- Add support for wildcards, e.g <stats>*</stats>
- Remove duplicate stats automatically
- Refactor peak summary to avoid separate arrays for stat attributes - everything is now wrapped up in a PeakStatInfo struct
- Modify CsvStats csv parsing to skip stats with only spaces in the name (for some reason LLM does this)
[FYI] ben.salem


#ROBOMERGE-SOURCE: CL 7669228 via CL 7673482
#ROBOMERGE-BOT: (v380-7672804)

[CL 7673532 by ben woodhouse in Main branch]
2019-07-31 13:44:37 -04:00
ben woodhouse
a7db48dff2 Make CSV metadata parsing robust to values with commas, in order to prevent exceptions crashes when reading [desktophwinfo].
A nicer solution would be to strip commas at source in the CSV profiler, but that won't work with existing data

[FYI] chris.bunner


#ROBOMERGE-SOURCE: CL 7650431 via CL 7654000
#ROBOMERGE-BOT: (v372-7473910)

[CL 7654072 by ben woodhouse in Main branch]
2019-07-29 17:35:39 -04:00
ben woodhouse
92669bc1d8 PerfReportTool 3.80: Fix edge-case exception when multiple events appear on the same frame as a loading screen event that we're stripping out. Looks like this is causing missing perf reports in some cases
[FYI] david.harvey, will.fissler


#ROBOMERGE-SOURCE: CL 6415835 via CL 6416661

[CL 6416675 by ben woodhouse in Main branch]
2019-05-10 20:30:24 -04:00