Commit Graph

3267 Commits

Author SHA1 Message Date
jerome delattre
4392f5f49e Gauntlet - Use Not-Run test state instead of In-Process to evaluate if a test pass need to be resumed
#rnx
#jira none
#rb none

#ROBOMERGE-SOURCE: CL 17252938 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17252959 by jerome delattre in ue5-release-engine-test branch]
2021-08-20 16:28:07 -04:00
jonathan adamczewski
05b7df7113 Various spelling & phrasing in comments & string
#jira none
##trivial

#ROBOMERGE-SOURCE: CL 17249885 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17249895 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-20 13:31:20 -04:00
geoff evans
055e9c2c25 IterationProfile Improvements
Call DDCCleanupCommandlet to tidy up the cache state so we don't get any cost/stat pollution during the profile
Set the UnusedFileAge to 0 to allow the DDCCleanupCommandlet to actually purge all ColdDerivedDataCache files
Set NODDCCLEANUP argument to prevent background cleanup of DDC data while we are profiling
Unset statnamedevents -- we don't track any of the legacy named events in telemetry, so let's not perpetuate it

#jira none
#rb trivial
#preflight 611d6c373a81b000013db0ad
#preflight 611d6c475e73720001dfe88e

#ROBOMERGE-SOURCE: CL 17233482 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17233495 by geoff evans in ue5-release-engine-test branch]
2021-08-19 10:01:26 -04:00
per larsson
2177cd7cec Zen: Parse package store manifest when staging from Zen
#rb CarlMagnus.Nordin
#rnx

#ROBOMERGE-OWNER: per.larsson
#ROBOMERGE-AUTHOR: per.larsson
#ROBOMERGE-SOURCE: CL 17230972 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)
#ROBOMERGE-CONFLICT from-shelf

[CL 17231508 by per larsson in ue5-release-engine-test branch]
2021-08-19 07:41:02 -04:00
dmytro vovk
cd4ffa251e Cache Turnkey veryfication results to reduce iteration times
#rb Nuno.Leiria
#jira UE-113865

#ROBOMERGE-SOURCE: CL 17223681 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17223690 by dmytro vovk in ue5-release-engine-test branch]
2021-08-18 18:06:57 -04:00
jerome delattre
dcc1d093a2 Gauntlet - avoid failing to produce a report if UE automation test json report is missing
#rnx
#rb trivial
#jira none

#ROBOMERGE-SOURCE: CL 17223043 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17223104 by jerome delattre in ue5-release-engine-test branch]
2021-08-18 17:38:25 -04:00
andrew davidson
e73db5f9e6 LWC: Large World Coordinates toggle defaults to ON.
Frosty and Fortnite default to OFF.

Preflight validation runs: 611c2b645e73720001a4ccf3, 611c318caabad10001091e20,611c2ba99db7df000111041b

#rb chad.garyet
#jira UE-121664
#preflight 611d33b23a81b00001365b33

#ROBOMERGE-SOURCE: CL 17218586 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17218602 by andrew davidson in ue5-release-engine-test branch]
2021-08-18 13:58:57 -04:00
nick darnell
bc8fdcbc5c AutomationTool - Logging out where the log file is when telling people to check the log file.
Andrew.Grant

#ROBOMERGE-SOURCE: CL 17206185 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17206189 by nick darnell in ue5-release-engine-test branch]
2021-08-17 17:21:32 -04:00
andrew davidson
f14e5dd9bc Undo //UE5/Main/... changelist 17203923
#ROBOMERGE-SOURCE: CL 17205235 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17205246 by andrew davidson in ue5-release-engine-test branch]
2021-08-17 16:28:05 -04:00
andrew davidson
96da376167 LWC: Large World Coordinates toggle defaults to ON.
Frosty and Fortnite default to OFF.

Preflight validation runs: 611aa00f3a81b00001d2998f, 611ad177aabad10001ca6a74, 611ad5773a81b00001d9947c, 611b8a2876f1fd00010eac6d

#rb ryan.durand
#jira UE-121664
#preflight 611b72d3aabad10001e5e3c3

#ROBOMERGE-SOURCE: CL 17203923 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17203933 by andrew davidson in ue5-release-engine-test branch]
2021-08-17 15:33:22 -04:00
jerome delattre
0829b8ec68 Gauntlet - Fix log not being parsed when no json report were generated during the test pass
#jira FORT-400491
#rnx
#rb none

#ROBOMERGE-SOURCE: CL 17183788 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17183797 by jerome delattre in ue5-release-engine-test branch]
2021-08-16 18:45:24 -04:00
jerome delattre
797899f2aa Gauntlet - fix a warning and failure to produce a report when ResumeOnCriticalFaiure is on and the test pass timeout.
#jira UEENGQA-38051
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 17177586 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17177594 by jerome delattre in ue5-release-engine-test branch]
2021-08-16 12:45:15 -04:00
jerome delattre
d0f6bf21ca Resume UE tests pass using json report as tracking support after test pass stoped because of critical failure.
* Implement loading Json report and resume from last know failure
* Make Gauntlet uses the retry state when critical failure occured on UE automation test.
* Capture the callstack if any and update the json report
* Attached passed log before retrying
* Use enum in json serialization and desirialization
* Use one common Event class for json report and parsing from the log
* Use an unique path for artifact at each runs/retries
* Use Json report to generate the pass summary in Gauntlet

#jira UEENGQA-38051
#rb Chris.Constantinescu, Eric.Knapick

#ROBOMERGE-SOURCE: CL 17160450 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17160467 by jerome delattre in ue5-release-engine-test branch]
2021-08-12 18:54:00 -04:00
jonathan adamczewski
6d74528a4b Update for changed MetaData.cs location
#jira none
#rb none
#trivial

#ROBOMERGE-SOURCE: CL 17152323 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17152326 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-12 10:57:45 -04:00
jonathan adamczewski
0251327c75 AutomationTool BuildAllScriptPlugins():
Clean up error text a little and add information so that when a project file fails to load it is easier to see who/what caused the project to load.

#jira none
#rb none

#ROBOMERGE-SOURCE: CL 17143264 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17143288 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-11 17:54:57 -04:00
jonathan adamczewski
267e840d27 AutomationTool:
When compiling script modules, log additional information if a project fails to load.
Correct path case for command line arguments

#jira none
#rb none

#ROBOMERGE-SOURCE: CL 17126379 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17126384 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-10 18:29:29 -04:00
joe kirchoff
fdc401e569 BenchmarkBuild: Save TaskTime in the event of an exception
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17124340 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17124361 by joe kirchoff in ue5-release-engine-test branch]
2021-08-10 17:00:30 -04:00
jonathan adamczewski
fda6a0dac3 AutomationTool:
Update references to DotNETCommon/MetaData.cs to Shared/MetaData.cs

#jira none
#rb none
#trivial

#ROBOMERGE-SOURCE: CL 17119587 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17119615 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-10 13:35:31 -04:00
jonathan adamczewski
e066d68356 AutomationTool:
Process "-Telemetry=path" explicitly as a global option.

#jira none
#rb none

#ROBOMERGE-SOURCE: CL 17119012 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17119019 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-10 13:04:32 -04:00
jonathan adamczewski
2b7b866f45 Move DotNETCommon/BuildUtilities to Shared/EpicGames.Build
Also moves DotNETCommon/Metadata.cs to Shared/Metadata.cs

#jira none
#rb tim.smith

#ROBOMERGE-SOURCE: CL 17116964 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17116977 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-10 11:09:18 -04:00
joe kirchoff
cbe8e1006b BenchmarkBuild: Record build time when task fails
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17108690 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17108741 by joe kirchoff in ue5-release-engine-test branch]
2021-08-09 18:24:52 -04:00
jonathan adamczewski
fa6f2a7b37 Gauntlet:
Skip the AutomationTool assembly: if Microsoft.Build.Framework can't be loaded, trying to do so will throw an exception.

#jira none
#rb Josh.Engebretson

#ROBOMERGE-SOURCE: CL 17108416 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17108448 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-09 18:00:15 -04:00
jonathan adamczewski
55fc07786f AutomationTool: make script-local copies of Newtonsoft.Json.dll
Neither AutomationTool, AutomationUtils, nor their dependencies have a reference to Newtonsoft.Json.dll, which can result in the dependency not being available.

Change the csproj files to make local copies of the dll (<Private>True</Private> means "Copy Local")

#jira none

#ROBOMERGE-SOURCE: CL 17105047 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17105059 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-09 13:27:05 -04:00
jonathan adamczewski
f270855eef AutomationTool: Compile script modules within the application
Add a layer of caching to avoid running msbuild as much as possible.

#jira UE-109181
#rb ben.marsh

#ROBOMERGE-SOURCE: CL 17102399 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17102408 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-09 10:39:35 -04:00
david harvey
c2d487e045 Adding RunUAT command to generate a report from DDPI values.
RunUAT ListPlatformDetails -DDPISection=ShaderPlatform/DataDrivenPlatformInfo


This generates an html report for DataDrivenPlatformInfo.ini files showing the values that are set & unset for all of them for easy comparison.


#jira UECON-467
#rb dave.barrett

#ROBOMERGE-SOURCE: CL 17101900 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17101910 by david harvey in ue5-release-engine-test branch]
2021-08-09 10:09:14 -04:00