- Modify defined tables to use the new attributes
#ROBOMERGE-OWNER: Ben.Woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16797223 via CL 16797225 via CL 16797229 via CL 16797233
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v836-16769935)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16804512 by ben woodhouse in ue5-release-engine-test branch]
This moves core data and summary table cache serialization away from the higher level summary table logic
#ROBOMERGE-OWNER: Ben.Woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16796939 via CL 16796941 via CL 16796946 via CL 16796948
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v836-16769935)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16804501 by ben woodhouse in ue5-release-engine-test branch]
- Split classes into separate files
- Make Summary types self-register so they're self-contained and more easily extendable
- Remove redundant XmlHelper class, convert existing usage of this to use the GetSafeAttribute method
- Output is unchanged (verified with unit tests)
#jira
#ROBOMERGE-OWNER: Ben.Woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16796904 via CL 16796908 via CL 16796909 via CL 16796910
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v836-16769935)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16804496 by ben woodhouse in ue5-release-engine-test branch]
- Add support for 3 levels of section barrier. Replace the bool "minor" attribute with int "level" attribute which can be 0,1 or 2 (0 is thickest)
- Fix bug where filter order is ignored for columns in the rowSort list in collated summary tables. Columns we're collating by will still appear first, but their relative order is preserved
- Adjust automated test summary table ordering to sort first by release version
#ROBOMERGE-OWNER: Ben.Woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16770651 via CL 16770653 via CL 16770656 via CL 16770657
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v836-16769935)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16770688 by ben woodhouse in ue5-release-engine-test branch]
- Add support for more than two sticky columns in collated summary tables
- Add format info for columns, replaces "lowIsBad" list. Allows specifying auto colorization rules and also numerical formatting
- Add support for multiple section boundaries, and minor (dashed line) section boundaries
- Section boundaries no longer require startToken or endToken (both are now optional)
- Section boundaries now can be specific to collated or full tables
#ROBOMERGE-OWNER: Ben.Woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 16740162 via CL 16740164 via CL 16740166 via CL 16740221
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v835-16672529)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16742906 by ben woodhouse in ue5-release-engine-test branch]
Update test data
[FYI] robert.millar
#ROBOMERGE-SOURCE: CL 16479765 via CL 16479776 via CL 16479781 via CL 16479784
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v823-16466674)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16479786 by ben woodhouse in ue5-release-engine-test branch]
- Don't specify colours for PeakSummary report type. Pass null instead and have it set them automatically (this also avoids the need to serialize)
#ROBOMERGE-OWNER: Ben.Woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16459036 via CL 16459039 via CL 16459051 via CL 16459069
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v818-16446889)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16459138 by ben woodhouse in ue5-release-engine-test branch]
- Refactor Peaks Summary to remove a lot of legacy code
- Sections can now be specified via summarySection elements in the XML
- Remove inMainSummary from graph elements
- Fix precision issue with SummaryTable data. Always store as doubles - never go via strings for numeric data
- NOTE: we are intentionally not bumping the SummaryTableCache version to avoid unnecessary churn here. Users can force an invalidate if necessary
- Pre-sort summary table to ensure deterministic output
- Update test results
#ROBOMERGE-OWNER: Ben.Woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16451878 via CL 16452760 via CL 16452854 via CL 16452934
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v818-16446889)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16459127 by ben woodhouse in ue5-release-engine-test branch]
Change 16308862 by ben.woodhouse[at]ROBOMERGE_FORTNITE_Dev_EngineMerge on 2021/05/12 22:00:30
PerfReportTool 4.47 - ExtraLinks summary
Adds a new ExtraLinks summary type, which reads a <csvfilename>.links file. The file contains lines with additional links in the format:
LongName,ShortName,URL
These links are added to a new Links section of the report. They are also added to summary table row, with the LongName as the key.
The section can be added to all reports by adding with -extraLinksSummary, or add the summary type "ExtraLinks" to the Report type XML.
#ROBOMERGE-SOURCE: CL 16308651 via CL 16309102 via CL 16315492
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v804-16311228)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 16315504 by ben woodhouse in ue5-release-engine-test branch]
- make -csvLinkRootPath work any type of path. Links are made relative to the passed-in csv directory and then the new root is added
#ROBOMERGE-SOURCE: CL 16178660 via CL 16178671 via CL 16178673 via CL 16208175
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v796-16191650)
[CL 16208185 by ben woodhouse in ue5-main branch]
-add -reportLinkRootPath arg to make report links relative to a particular location
-add -csvLinkRootPath arg to make CSV links relative to a particular location
-add -listFiles arg to just list files that pass the filter instead of processing them
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 16177813 via CL 16177817 via CL 16177818 via CL 16178057
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v792-16166999)
[CL 16178065 by ben woodhouse in ue5-main branch]
- Add support for >,<,>=,<= operators for metadata queries (for numerical fields)
- Use doubles for SummaryTable/SummaryTableColumn values, so we can accurately show display values (does not affect summary cache - this was already at double precision)
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16110876 via CL 16110877 via CL 16110881 via CL 16110882
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)
[CL 16117137 by ben woodhouse in ue5-main branch]
- Add -requireMetadata option to ignore CSVs without metadata
- Clean up help text so the lines arent too long for standard console output
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16105233 via CL 16105248 via CL 16105254 via CL 16105255
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)
[CL 16116861 by ben woodhouse in ue5-main branch]
- add -summaryTableCacheUseOnlyCsvID param to ignore the ReportType hash and just use the CSV ID for summaryTableCache ID entries. This is a good idea if you want these files to persist and you don't want them to be regenerated when report types change
- Add ReportType and ReportTypeID to the summaryTable ToolMetaData entries
- Use the reportType name as part of the report hash (this will invalidate existing cache files!)
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16024419 via CL 16024452 via CL 16024531 via CL 16045954
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)
[CL 16116846 by ben woodhouse in ue5-main branch]
- Refactor code to avoid use of the term Metadata to refer to summary table row data. The term metadata was easily confused with CSV metadata, and in most cases, it wasn't actually metadata.
- This refactor resulted in some minor breaking changes in XML format and command line params. The affected reports and scripts have been fixed up accordingly
The following were renamed:
SummaryMetadata -> SummaryTableRowData
SummaryMetadataValue->SummaryTableElement
SummaryMetadataTable->SummaryTable
metadataKey -> summaryStatName (also affects XML)
metadataPrefix -> summaryStatNamePrefix (also affects XML)
metadataCache -> summaryTableCache
Commandline params, e.g: -metadataCache -> -summaryMetadataCache
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16023141 via CL 16023153 via CL 16023173 via CL 16045518
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)
[CL 16116817 by ben woodhouse in ue5-main branch]
-5x faster metadata cache only reads:
- Only read the initial CSV metadata entries before metadata filtering (since these always appear first)
- Skip report type determiniation in metadataonly mode (this is slow and unnecessary)
-crash fix (don't crash if old version metadata is encountered, just skip and log)
Querying a single device profile across 31k CSVs of mixed device profiles takes 2.1s instead of 10.2s previously
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16015980 via CL 16015986 via CL 16015993 via CL 16043535
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)
[CL 16116807 by ben woodhouse in ue5-main branch]
Open metadata and .bin files readonly
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16013849 via CL 16013851 via CL 16013859 via CL 16043212
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)
[CL 16116794 by ben woodhouse in ue5-main branch]
- Add -metadataCacheIn mode for running without CSVs
- Use a BlockingCollection to eliminate precache threading bottleneck and simplify precache thread producer/consumer code
- Threading still hits a performance bottleneck at around 6 threads when running metadata queries currently, and gets slower after that as more threads are added.
- Need further work to optimize GetReportTypeXML (too much unnecessary linq) and SummaryMetadata::TryReadFromCache
#ROBOMERGE-OWNER: ben.woodhouse
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 15974021 via CL 15974024 via CL 15974028 via CL 16027282
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)
[CL 16112788 by ben woodhouse in ue5-main branch]
