Commit Graph

17 Commits

Author SHA1 Message Date
geoff evans
96039ea7a5 IterationProfile: Roll back into a single node to fix some errors
#jira none
#rb trivial

[CL 16768700 by geoff evans in ue5-main branch]
2021-06-24 00:32:57 -04:00
geoff evans
c30333de4b IterationProfile: Add dependency of Cold onto Warm, only if Cold is run.
#jira none
#rb trivial

[CL 16764759 by geoff evans in ue5-main branch]
2021-06-23 18:41:28 -04:00
geoff evans
80ba54d90f IterationProfile: Increase maxduration to 6 hours
#jira none
#rb trivial

[CL 16760162 by geoff evans in ue5-main branch]
2021-06-23 14:20:50 -04:00
geoff evans
1f56291157 IterationProfile: Split Code and Warm into their own Node so they are easier to eyeball the exec times of in the Horde web client.
This was tested via a local IterationProfile run.

#jira none
#rb trivial

[CL 16751778 by geoff evans in ue5-main branch]
2021-06-22 19:47:50 -04:00
geoff evans
d4e053aac9 IterationProfile: Promote ShaderCompiler logging to Verbose so we don't time out building shaders for 30+ minutes
#jira none
#rb jerome.delattre
#preflight 60d25dfca81904000153eb16

[CL 16751129 by geoff evans in ue5-main branch]
2021-06-22 19:06:46 -04:00
Jerome Delattre
451cb28c1d Fix Label category
#rnx
#jira none
#rb none

[CL 16698128 by Jerome Delattre in ue5-main branch]
2021-06-16 18:51:40 -04:00
Ryan Durand
594cd2717b Removing TestWin64 as a valid type for IterationProfile.xml.
#rnx
#rb none
#jira none

[CL 16488037 by Ryan Durand in ue5-main branch]
2021-05-27 13:37:59 -04:00
Ryan Durand
d1947b6afa Consolidating IterationTest to a single agent and workspace type until we determine if this fixes things.
#rnx
#rb none
#jira none
#fyi chris.cossiah

[CL 16487637 by Ryan Durand in ue5-main branch]
2021-05-27 13:12:25 -04:00
geoff evans
f024fc1464 IterationProfile polish
Update TotalSeconds -> TotalDurationSeconds to make it unambiguously a time span (vs. time stamp)
Add some Telemetry points for Reverb PIE stalls
Don't push telemetry from Preflight builds

Tested with local UAT runs

#jira none
#rb trivial

[CL 16413783 by geoff evans in ue5-main branch]
2021-05-20 21:13:42 -04:00
geoff evans
ce2fe10a6a IterationProfile: Track Map_Load on Telemetry
Disable loading a map at startup to a) remove the map load from EditorInit, and b) make the automation plugin's map load the first, making it precisely tracked through Telemetry

Tested with local UAT runs.

#jira none
#rb trivial
#fyi francis.hurteau

[CL 16410566 by geoff evans in ue5-main branch]
2021-05-20 17:25:16 -04:00
geoff evans
a1556b13ae IterationProfile: Support for baseline comparison of tracked telemetry.
The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions.

Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared.
Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build.
Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage.
Restructure CSV loading to pull some complexity out of the Commandlet::Main.
The baseline telemetry csvs included in this change are from @16364136.

#jira UETOOL-3433
#rb tbd
#preflight 60a46e1565d94d0001225cfe

[CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
geoff evans
a055d11fa5 Rename Win64 and MacOS support arguments.
Preliminary change for bless support. These arguments are general platform toggles now.

#rb none

[CL 16337281 by geoff evans in ue5-main branch]
2021-05-14 18:34:21 -04:00
geoff evans
654bba3c0c IterationProfile: Add explicit test name parameter to the commandlet, and specify exact test name w/o project or platform to keep telemetry concise.
#jira none
#rb jerome.delattre

[CL 16332651 by geoff evans in ue5-main branch]
2021-05-14 14:24:21 -04:00
geoff evans
ea46919074 Fix relative pathing issue
#rb trivial

[CL 16321116 by geoff evans in ue5-main branch]
2021-05-13 18:45:22 -04:00
geoff evans
dae79a1660 Parallelize Run phase of Iteration Profile
#rb trivial

[CL 16320052 by geoff evans in ue5-main branch]
2021-05-13 17:52:04 -04:00
geoff evans
b7adbc6730 Support for running multiple projects within the same Iteration Profile build
* This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue.

AutomationTool:
* Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>)

Horde:
* Put project name into labels to isolate trends graph as builds are run on different projects

#jira none
#rb tbd

[CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
geoff evans
d0d74aa4d0 Move IterationProfile out of Restricted
#jira none
#rb trivial

[CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00