izzy/UnrealEngineUWP
0
0
Fork 0
mirror of https://github.com/izzy2lost/UnrealEngineUWP.git synced 2026-03-26 18:15:20 -07:00
Code Issues Packages Projects Releases Wiki Activity
Files
989b9d01cf211d96d486de4e5eff03475b02f6ca
UnrealEngineUWP/Engine/Build/IterationProfile.xml

253 lines
15 KiB
XML
Raw Normal View History

Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<?xml version='1.0' ?>
<BuildGraph xmlns="http://www.epicgames.com/BuildGraph" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.epicgames.com/BuildGraph ../../../Build/Graph/Schema.xsd" >
<!-- Horde Options -->
<EnvVar Name="UE_HORDE_JOBID"/>
<Option Name="Horde" Restrict="true|false" DefaultValue="false" Description="Whether the build is running under Horde"/>
<Option Name="PreflightChange" DefaultValue="" Description="Preflight CL number if preflight, empty otherwise"/>
<Option Name="SetPreflightFlag" DefaultValue="true" Description="Whether to set the IsPreflight flag to true for preflights. Use with caution: this will cause things like build versions to be set to their non-preflight form."/>
<Property Name="IsPreflight" Value="false"/>
<Property Name="IsPreflight" Value="true" If="'$(PreflightChange)' != '' And $(SetPreflightFlag)"/>
<Property Name="PreflightSuffix" Value=""/>
<Property Name="PreflightSuffix" Value="-PF-$(PreflightChange)-$(UE_HORDE_JOBID)" If="$(IsPreflight)"/>
<Property Name="BuildName" Value="$(EscapedBranch)-CL-$(Change)$(PreflightSuffix)"/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<!-- Projects setup-->
<Option Name="ProjectNames" DefaultValue="false" Restrict=".+" Description="The Projects to test"/>
<Error Message="No project names specified" If="'$(ProjectNames)' == 'false'"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Option Name="UgsProject" DefaultValue="" Description="The relative path to the project"/>
<Option Name="UgsBadge" DefaultValue="" Description="The badge name to use with labels"/>
<!-- Platform Options -->
Rename Win64 and MacOS support arguments. Preliminary change for bless support. These arguments are general platform toggles now. #rb none [CL 16337281 by geoff evans in ue5-main branch]
2021-05-14 18:34:21 -04:00
<Option Name="Win64" DefaultValue="false" Description="Enable Win64 Platform"/>
<Option Name="MacOS" DefaultValue="false" Description="Enable MacOS Platform"/>
<Error Message="No platforms specified" If="!$(Win64) And !$(MacOS)"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<!-- Build Options -->
<Property Name="ExtraPluginsCommaDelims" Value="EditorTests"/>
<Property Name="ExtraPluginsPlusDelims" Value=""/> <!-- populated programmatically -->
<ForEach Name="Plugin" Values="$(ExtraPluginsCommaDelims)">
<Property Name="ExtraPluginString" Value="$(Plugin)" If="'$(ExtraPluginsPlusDelims)' == ''"/>
<Property Name="ExtraPluginString" Value="$(ExtraPluginsPlusDelims)+$(Plugin)" If="'$(ExtraPluginsPlusDelims)' != ''"/>
<Property Name="ExtraPluginsPlusDelims" Value="$(ExtraPluginString)"/>
</ForEach>
<Property Name="EditorCompileArgs" Value="-BuildPlugin=$(ExtraPluginsPlusDelims)"/>
<!-- Automation uses its own network output directory -->
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="AutomationNetworkRootDir" Value="//epicgames.net/root/Builds/Automation/Reports/IterationProfile"/>
<Property Name="AutomationNetworkRootDir" Value="/Volumes/Builds/Automation/Reports/IterationProfile" If="'$(HostPlatform)' != 'Win64'"/>
<Property Name="AutomationReportRootUrl" Value="http://automation.epicgames.net/reports/iterationprofile"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Property Name="AutomationLocalOutputDir" Value="$(RootDir)/LocalBuilds/Reports"/>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<!-- Outputs for this run -->
<Property Name="AutomationNetworkOutputDir" Value="$(AutomationNetworkRootDir)/$(BuildName)"/>
<Property Name="AutomationReportBaseUrl" Value="$(AutomationReportRootUrl)/$(BuildName)"/>
<Property Name="AutomationOutputDir" Value="$(AutomationLocalOutputDir)"/>
<Property Name="AutomationOutputDir" Value="$(AutomationNetworkOutputDir)" If="$(IsBuildMachine)"/>
<!-- Options for baselining -->
<Option Name="RunAndBaseline" DefaultValue="false" Description="Enable graph dependencies for running and setting the baseline in a single build."/>
<Option Name="BaselineDir" DefaultValue="$(RootDir)/Engine/Build/Baseline" Description="The revision controlled location of the baseline telemetry files."/>
<Option Name="BaselineBuildName" DefaultValue="$(BuildName)" Description="The BuildName of the build to set as the new baseline to compare against."/>
<Property Name="BaselineAutomationNetworkOutputDir" Value="$(AutomationNetworkRootDir)/$(BaselineBuildName)"/>
<Property Name="BaselineAutomationNetworkOutputDir" Value="$(AutomationOutputDir)" If="$(RunAndBaseline)"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<!-- Arguments for tracing -->
<Option Name="ColdPass" DefaultValue="true" Description="Perform cold workspace loading profiles"/>
<Option Name="ColdArgs" DefaultValue="-ddc=Cold" Description="Arguments to unreal for the cold pass"/>
<Option Name="WarmPass" DefaultValue="true" Description="Perform warm (incremental) workspace loading profiles"/>
<Option Name="WarmArgs" DefaultValue="-ddc=Warm" Description="Arguments to unreal for the warm pass"/>
<Property Name="TraceDir" Value="$(RootDir)/Engine/Programs/UnrealInsights/Saved/TraceSessions"/>
<!-- Gauntlet arguments for all runs -->
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<Property Name="GauntletArgs" Value="-build=editor -UseEditor -verbose -report -tempdir=&quot;$(RootDir)/Tests&quot;"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Property Name="GauntletArgs" Value="$(GauntletArgs) -branch=$(Branch) -ECBranch=&quot;$(Branch)&quot; -ECChangelist=&quot;$(Change)&quot;"/>
<Property Name="GauntletArgs" Value="$(GauntletArgs) -maxduration=14400"/>
<!-- Gauntlet arguments for profile runs -->
<Property Name="ProfileArgs" Value="-test=EditorTests -testfilter=Group:Iteration -tracefiletrunc"/>
<Property Name="ProfileArgs" Value="$(ProfileArgs) -EnablePlugins=$(ExtraPluginsCommaDelims)" If="'$(ExtraPluginsCommaDelims)' != ''"/>
<Property Name="ProfileArgs" Value="$(ProfileArgs) -TreatLogWarningsAsTestErrors=false -TreatLogErrorsAsTestErrors=false"/>
<Property Name="ProfileArgs" Value="$(ProfileArgs) -GameInstanceLostTimerSeconds=3600"/>
IterationProfile: Track Map_Load on Telemetry Disable loading a map at startup to a) remove the map load from EditorInit, and b) make the automation plugin's map load the first, making it precisely tracked through Telemetry Tested with local UAT runs. #jira none #rb trivial #fyi francis.hurteau [CL 16410566 by geoff evans in ue5-main branch]
2021-05-20 17:25:16 -04:00
<Property Name="ProfileArgs" Value="$(ProfileArgs) -NoLoadLevelAtStartup"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Property Name="ProfileArgs" Value="$(ProfileArgs) -NoShaderDistrib"/>
<!-- Gauntlet arguments for summarize commandlet runs -->
Fix relative pathing issue #rb trivial [CL 16321116 by geoff evans in ue5-main branch]
2021-05-13 18:45:22 -04:00
<Property Name="SummarizeArgs" Value="-statsfile=&quot;$(RootDir)/Engine/Build/IterationProfileStats.csv&quot;"/>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="SummarizeArgs" Value="$(SummarizeArgs) -skipbaseline" If="$(RunAndBaseline)"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<!-- The main test running macro logic that is expanded with platform and dependency node names -->
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Macro Name="RunTest" Arguments="OutputName;PlatformName;ProjectName;NodeName;Phase;Args">
IterationProfile: Add explicit test name parameter to the commandlet, and specify exact test name w/o project or platform to keep telemetry concise. #jira none #rb jerome.delattre [CL 16332651 by geoff evans in ue5-main branch]
2021-05-14 14:24:21 -04:00
<Property Name="TestName" Value="IterationProfile$(Phase)"/>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="TraceFilePath" Value="$(TraceDir)/$(ProjectName)$(PlatformName)$(TestName).utrace"/>
<Property Name="TelemetryFileName" Value="$(ProjectName)$(PlatformName)$(TestName)Telemetry.csv"/>
<Property Name="TelemetryFilePath" Value="$(TraceDir)/$(TelemetryFileName)"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Property Name="ReportExportPath" Value="$(AutomationOutputDir)/$(OutputName)"/>
<Property Name="ReportParams" Value="-ReportExportPath=&quot;$(ReportExportPath)&quot;"/>
<Do If="$(IsBuildMachine)">
<Property Name="ReportExportURL" Value="$(AutomationReportBaseUrl)/$(OutputName)/index.html"/>
<Property Name="ReportParams" Value="$(ReportParams) -ReportURL=&quot;$(ReportExportURL)&quot; -WriteTestResultsForHorde -HordeTestDataKey=&quot;$(NodeName) $(Phase)$(PreflightSuffix)&quot;"/>
</Do>
<Property Name="LogOutputParams" Value="-logdir=&quot;$(AutomationOutputDir)/$(OutputName)&quot;"/>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="AllGauntletArgs" Value="-project=$(ProjectName) -platform=$(PlatformName) $(GauntletArgs) $(ReportParams) $(LogOutputParams) -artifactname=&quot;$(Phase)&quot;"/>
<!-- Run the Gauntlet Iteration Profile Tests -->
<Command Name="RunUnreal" Arguments="$(AllGauntletArgs) $(ProfileArgs) $(Args) -tracefile=&quot;$(TraceFilePath)&quot;"/>
<!-- Translate the utrace file into csv data -->
<Commandlet Name="SummarizeTrace" Arguments="-NoEnginePlugins -inputfile=&quot;$(TraceFilePath)&quot; -testname=&quot;$(TestName)&quot; $(SummarizeArgs)"/>
<!-- Stash the telemetry file in case this build will become the baseline in the future -->
<Copy From="$(TelemetryFilePath)" To="$(AutomationOutputDir)/$(TelemetryFileName)"/>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Do If="$(IsBuildMachine)">
<!-- Files copied to this folder are automatically uploaded to Horde as artifacts -->
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Copy From="$(TraceDir)/$(ProjectName)$(PlatformName)$(TestName)*.*" To="$(RootDir)/Engine/Programs/AutomationTool/Saved/Logs"/>
IterationProfile polish Update TotalSeconds -> TotalDurationSeconds to make it unambiguously a time span (vs. time stamp) Add some Telemetry points for Reverb PIE stalls Don't push telemetry from Preflight builds Tested with local UAT runs #jira none #rb trivial [CL 16413783 by geoff evans in ue5-main branch]
2021-05-20 21:13:42 -04:00
<Do If="!$(IsPreflight)">
<!-- Publish telemetry to the db -->
<Command Name="PublishUnrealAutomationTelemetry" Arguments="-CSVFile=&quot;$(TelemetryFilePath)&quot; -project=$(ProjectName) -changelist=$(Change) -branch=$(Branch)"/>
</Do>
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
</Do>
</Macro>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<!-- This will be set to the names of all nodes we want to execute based on specified platforms/options -->
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="RunGraphNodes" Value=""/>
<!-- Mimic of the Runner that retrieves the stashed telemetry file and copies it to the depot location -->
<Macro Name="BaselineResult" Arguments="PlatformName;ProjectName;Phase">
<Property Name="TestName" Value="IterationProfile$(Phase)"/>
<Property Name="TelemetryFileName" Value="$(ProjectName)$(PlatformName)$(TestName)Telemetry.csv"/>
<Property Name="LocalTelemetryFilePath" Value="$(BaselineDir)/$(TelemetryFileName)"/>
<Property Name="NetworkTelemetryFilePath" Value="$(BaselineAutomationNetworkOutputDir)/$(TelemetryFileName)"/>
<Copy From="$(NetworkTelemetryFilePath)" To="$(LocalTelemetryFilePath)" Overwrite="true"/>
<Do If="!$(IsBuildMachine)">
<Spawn Exe="p4" Arguments="reconcile -ea $(LocalTelemetryFilePath)"/>
</Do>
</Macro>
<!-- This will be set to the names of all nodes we want to execute based on specified platforms/options -->
<Property Name="BaselineGraphNodes" Value=""/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<!-- Windows -->
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Agent Name="Prerequisites Agent Win64" Type="Win64">
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<Node Name="Compile Prerequisites Win64Host">
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Compile Target="UnrealHeaderTool" Platform="Win64" Configuration="Development"/>
<Compile Target="ShaderCompileWorker" Platform="Win64" Configuration="Development"/>
<Compile Target="CrashReportClient" Platform="Win64" Configuration="Shipping"/>
<Compile Target="CrashReportClientEditor" Platform="Win64" Configuration="Shipping"/>
</Node>
</Agent>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<!-- macOS -->
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Agent Name="Prerequisites Agent Mac" Type="TestMac">
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<Node Name="Compile Prerequisites Mac">
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
<Compile Target="UnrealHeaderTool" Platform="Mac" Configuration="Development"/>
<Compile Target="ShaderCompileWorker" Platform="Mac" Configuration="Development"/>
<Compile Target="CrashReportClient" Platform="Mac" Configuration="Shipping"/>
<Compile Target="CrashReportClientEditor" Platform="Mac" Configuration="Shipping"/>
</Node>
</Agent>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<!-- ForEach Project to run, generate the graph nodes -->
<ForEach Name="ProjectName" Values="$(ProjectNames)">
<!-- Windows -->
<Agent Name="Prerequisites Agent Win64" Type="Win64">
<Node Name="Compile $(ProjectName) Editor Win64" Requires="Compile Prerequisites Win64Host">
<Compile Target="$(ProjectName)Editor" Platform="Win64" Configuration="Development" Arguments="$(EditorCompileArgs)"/>
</Node>
</Agent>
Removing TestWin64 as a valid type for IterationProfile.xml. #rnx #rb none #jira none [CL 16488037 by Ryan Durand in ue5-main branch]
2021-05-27 13:37:59 -04:00
<Agent Name="Automated Test Agent $(ProjectName) Win64" Type="IterationTest">
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="PlatformName" Value="Win64"/>
<Property Name="OutputName" Value="$(ProjectName)$(PlatformName)IterationProfile"/>
<Property Name="RunNodeName" Value="Run $(OutputName)"/>
<Property Name="RunNodeRequires" Value="Compile Prerequisites $(PlatformName)Host;Compile $(ProjectName) Editor $(PlatformName)"/>
<Property Name="BaselineNodeName" Value="Baseline $(OutputName)"/>
<Property Name="BaselineNodeRequires" Value=""/>
<Property Name="BaselineNodeRequires" Value="$(RunNodeName)" If="$(RunAndBaseline)"/>
Rename Win64 and MacOS support arguments. Preliminary change for bless support. These arguments are general platform toggles now. #rb none [CL 16337281 by geoff evans in ue5-main branch]
2021-05-14 18:34:21 -04:00
<Do If="$(Win64)">
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<!-- Run the Tests -->
<Node Name="$(RunNodeName)" Requires="$(RunNodeRequires)">
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<Do If="$(ColdPass)">
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Expand Name="RunTest" OutputName="$(OutputName)" PlatformName="$(PlatformName)" ProjectName="$(ProjectName)" NodeName="$(RunNodeName)" Phase="Cold" Args="$(ColdArgs)"/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
</Do>
<Do If="$(WarmPass)">
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Expand Name="RunTest" OutputName="$(OutputName)" PlatformName="$(PlatformName)" ProjectName="$(ProjectName)" NodeName="$(RunNodeName)" Phase="Warm" Args="$(WarmArgs)"/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
</Do>
</Node>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="RunGraphNodes" Value="$(RunGraphNodes);$(RunNodeName)"/>
<!-- Baseline the Results -->
<Node Name="$(BaselineNodeName)" Requires="$(BaselineNodeRequires)">
<Do If="$(ColdPass)">
<Expand Name="BaselineResult" PlatformName="$(PlatformName)" ProjectName="$(ProjectName)" Phase="Cold"/>
</Do>
<Do If="$(WarmPass)">
<Expand Name="BaselineResult" PlatformName="$(PlatformName)" ProjectName="$(ProjectName)" Phase="Warm"/>
</Do>
</Node>
<Property Name="BaselineGraphNodes" Value="$(BaselineGraphNodes);$(BaselineNodeName)"/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
</Do>
</Agent>
<!-- macOS -->
<Agent Name="Prerequisites Agent Mac" Type="TestMac">
<Node Name="Compile $(ProjectName) Editor Mac" Requires="Compile Prerequisites Mac">
<Compile Target="$(ProjectName)Editor" Platform="Mac" Configuration="Development" Arguments="$(EditorCompileArgs)"/>
</Node>
</Agent>
Parallelize Run phase of Iteration Profile #rb trivial [CL 16320052 by geoff evans in ue5-main branch]
2021-05-13 17:52:04 -04:00
<Agent Name="Automated Test Agent $(ProjectName) Mac" Type="TestMac">
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="PlatformName" Value="Mac"/>
<Property Name="OutputName" Value="$(ProjectName)$(PlatformName)IterationProfile"/>
<Property Name="RunNodeName" Value="Run $(OutputName)"/>
<Property Name="RunNodeRequires" Value="Compile Prerequisites $(PlatformName);Compile $(ProjectName) Editor $(PlatformName)"/>
<Property Name="BaselineNodeName" Value="Baseline $(OutputName)"/>
<Property Name="BaselineNodeRequires" Value=""/>
<Property Name="BaselineNodeRequires" Value="$(RunNodeName)" If="$(RunAndBaseline)"/>
Rename Win64 and MacOS support arguments. Preliminary change for bless support. These arguments are general platform toggles now. #rb none [CL 16337281 by geoff evans in ue5-main branch]
2021-05-14 18:34:21 -04:00
<Do If="$(MacOS)">
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<!-- Run the Tests -->
<Node Name="$(RunNodeName)" Requires="$(RunNodeRequires)">
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
<Do If="$(ColdPass)">
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Expand Name="RunTest" OutputName="$(OutputName)" PlatformName="$(PlatformName)" ProjectName="$(ProjectName)" NodeName="$(RunNodeName)" Phase="Cold" Args="$(ColdArgs)"/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
</Do>
<Do If="$(WarmPass)">
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Expand Name="RunTest" OutputName="$(OutputName)" PlatformName="$(PlatformName)" ProjectName="$(ProjectName)" NodeName="$(RunNodeName)" Phase="Warm" Args="$(WarmArgs)"/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
</Do>
</Node>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<Property Name="RunGraphNodes" Value="$(RunGraphNodes);$(RunNodeName)"/>
<!-- Baseline the Results -->
<Node Name="$(BaselineNodeName)" Requires="$(BaselineNodeRequires)">
<Do If="$(ColdPass)">
<Expand Name="BaselineResult" PlatformName="$(PlatformName)" ProjectName="$(ProjectName)" Phase="Cold"/>
</Do>
<Do If="$(WarmPass)">
<Expand Name="BaselineResult" PlatformName="$(PlatformName)" ProjectName="$(ProjectName)" Phase="Warm"/>
</Do>
</Node>
<Property Name="BaselineGraphNodes" Value="$(BaselineGraphNodes);$(BaselineNodeName)"/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
</Do>
</Agent>
<!-- Graph nodes that really tie the room together -->
Parallelize Run phase of Iteration Profile #rb trivial [CL 16320052 by geoff evans in ue5-main branch]
2021-05-13 17:52:04 -04:00
<Label Category="Tests" Name="$(ProjectName) Iteration Profile Win64" Requires="Prerequisites Agent Win64;Automated Test Agent $(ProjectName) Win64" UgsProject="$(UgsProject)" UgsBadge="$(UgsBadge)"/>
<Label Category="Tests" Name="$(ProjectName) Iteration Profile Mac" Requires="Prerequisites Agent Mac;Automated Test Agent $(ProjectName) Mac" UgsProject="$(UgsProject)" UgsBadge="$(UgsBadge)"/>
Support for running multiple projects within the same Iteration Profile build * This will give us wider data collection to detect regressions in the cases where a single project has a blocking issue. AutomationTool: * Add support for appending semicolon delimited parameters into Options (to feed <ForEach/>) Horde: * Put project name into labels to isolate trends graph as builds are run on different projects #jira none #rb tbd [CL 16318721 by geoff evans in ue5-main branch]
2021-05-13 16:53:24 -04:00
</ForEach>
IterationProfile: Support for baseline comparison of tracked telemetry. The baseline telemetry csv files are an artifact of a build that are stashed in revision control used to compare telemetry values against bleeding edge builds to detect regressions. Add support to IterationProfile.xml for stashing and retrieving a specific build's telemetry data for blessing as the baseline for which builds are compared. Add SummarizeTraceCommandlet support to ingest the blessed telemetry csv for comparison against the current build. Add warning and error threshold columns to stats.csv, and add support for percentage and cardinal value ascending overage and descending underage. Restructure CSV loading to pull some complexity out of the Commandlet::Main. The baseline telemetry csvs included in this change are from @16364136. #jira UETOOL-3433 #rb tbd #preflight 60a46e1565d94d0001225cfe [CL 16405602 by geoff evans in ue5-main branch]
2021-05-20 12:47:21 -04:00
<!-- These Nodes run the Editor and gather profile data -->
<Aggregate Name="Run Iteration Profile Tests" Requires="$(RunGraphNodes)"/>
<!-- These Nodes copies the CSV telemetry files to special location in the depot, thus making them the "baselines" -->
<Aggregate Name="Baseline Iteration Profile Results" Requires="$(BaselineGraphNodes)"/>
<!-- This Node submits the baseline CSV files to Perforce -->
<!--
<Agent Name="Submission Agent Win64" Type="Win64">
<Node Name="Submit Baseline To Perforce" Requires="$(BaselineGraphNodes)">
<Submit Description="[CL $(Change)] Updated IterationProfile baselines" Files="$(BaselineFiles)"/>
</Node>
</Agent>
-->
Move IterationProfile out of Restricted #jira none #rb trivial [CL 16316184 by geoff evans in ue5-main branch]
2021-05-13 14:36:41 -04:00
</BuildGraph>
Reference in New Issue Copy Permalink