Commit Graph

3257 Commits

Author SHA1 Message Date
jerome delattre
0829b8ec68 Gauntlet - Fix log not being parsed when no json report were generated during the test pass
#jira FORT-400491
#rnx
#rb none

#ROBOMERGE-SOURCE: CL 17183788 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17183797 by jerome delattre in ue5-release-engine-test branch]
2021-08-16 18:45:24 -04:00
jerome delattre
797899f2aa Gauntlet - fix a warning and failure to produce a report when ResumeOnCriticalFaiure is on and the test pass timeout.
#jira UEENGQA-38051
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 17177586 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17177594 by jerome delattre in ue5-release-engine-test branch]
2021-08-16 12:45:15 -04:00
jerome delattre
d0f6bf21ca Resume UE tests pass using json report as tracking support after test pass stoped because of critical failure.
* Implement loading Json report and resume from last know failure
* Make Gauntlet uses the retry state when critical failure occured on UE automation test.
* Capture the callstack if any and update the json report
* Attached passed log before retrying
* Use enum in json serialization and desirialization
* Use one common Event class for json report and parsing from the log
* Use an unique path for artifact at each runs/retries
* Use Json report to generate the pass summary in Gauntlet

#jira UEENGQA-38051
#rb Chris.Constantinescu, Eric.Knapick

#ROBOMERGE-SOURCE: CL 17160450 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17160467 by jerome delattre in ue5-release-engine-test branch]
2021-08-12 18:54:00 -04:00
jonathan adamczewski
6d74528a4b Update for changed MetaData.cs location
#jira none
#rb none
#trivial

#ROBOMERGE-SOURCE: CL 17152323 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17152326 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-12 10:57:45 -04:00
jonathan adamczewski
0251327c75 AutomationTool BuildAllScriptPlugins():
Clean up error text a little and add information so that when a project file fails to load it is easier to see who/what caused the project to load.

#jira none
#rb none

#ROBOMERGE-SOURCE: CL 17143264 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17143288 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-11 17:54:57 -04:00
jonathan adamczewski
267e840d27 AutomationTool:
When compiling script modules, log additional information if a project fails to load.
Correct path case for command line arguments

#jira none
#rb none

#ROBOMERGE-SOURCE: CL 17126379 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17126384 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-10 18:29:29 -04:00
joe kirchoff
fdc401e569 BenchmarkBuild: Save TaskTime in the event of an exception
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17124340 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17124361 by joe kirchoff in ue5-release-engine-test branch]
2021-08-10 17:00:30 -04:00
jonathan adamczewski
fda6a0dac3 AutomationTool:
Update references to DotNETCommon/MetaData.cs to Shared/MetaData.cs

#jira none
#rb none
#trivial

#ROBOMERGE-SOURCE: CL 17119587 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17119615 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-10 13:35:31 -04:00
jonathan adamczewski
e066d68356 AutomationTool:
Process "-Telemetry=path" explicitly as a global option.

#jira none
#rb none

#ROBOMERGE-SOURCE: CL 17119012 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17119019 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-10 13:04:32 -04:00
jonathan adamczewski
2b7b866f45 Move DotNETCommon/BuildUtilities to Shared/EpicGames.Build
Also moves DotNETCommon/Metadata.cs to Shared/Metadata.cs

#jira none
#rb tim.smith

#ROBOMERGE-SOURCE: CL 17116964 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17116977 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-10 11:09:18 -04:00
joe kirchoff
cbe8e1006b BenchmarkBuild: Record build time when task fails
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17108690 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17108741 by joe kirchoff in ue5-release-engine-test branch]
2021-08-09 18:24:52 -04:00
jonathan adamczewski
fa6f2a7b37 Gauntlet:
Skip the AutomationTool assembly: if Microsoft.Build.Framework can't be loaded, trying to do so will throw an exception.

#jira none
#rb Josh.Engebretson

#ROBOMERGE-SOURCE: CL 17108416 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17108448 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-09 18:00:15 -04:00
jonathan adamczewski
55fc07786f AutomationTool: make script-local copies of Newtonsoft.Json.dll
Neither AutomationTool, AutomationUtils, nor their dependencies have a reference to Newtonsoft.Json.dll, which can result in the dependency not being available.

Change the csproj files to make local copies of the dll (<Private>True</Private> means "Copy Local")

#jira none

#ROBOMERGE-SOURCE: CL 17105047 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v854-17104634)

[CL 17105059 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-09 13:27:05 -04:00
jonathan adamczewski
f270855eef AutomationTool: Compile script modules within the application
Add a layer of caching to avoid running msbuild as much as possible.

#jira UE-109181
#rb ben.marsh

#ROBOMERGE-SOURCE: CL 17102399 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17102408 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-09 10:39:35 -04:00
david harvey
c2d487e045 Adding RunUAT command to generate a report from DDPI values.
RunUAT ListPlatformDetails -DDPISection=ShaderPlatform/DataDrivenPlatformInfo


This generates an html report for DataDrivenPlatformInfo.ini files showing the values that are set & unset for all of them for easy comparison.


#jira UECON-467
#rb dave.barrett

#ROBOMERGE-SOURCE: CL 17101900 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17101910 by david harvey in ue5-release-engine-test branch]
2021-08-09 10:09:14 -04:00
geoff evans
d38f02ab35 IterationProfile: Add Trace Bookmark support
* Add bookmark pair to create telemetry scope for LoadAssetAndPIE, whitelist it in project-specific statistics files
* Split Stats.csv files into global Engine/Build/IterationProfile.csv and per-project Build/IterationProfile.csv directives (for project privacy)
* Global statistics file is now assumed, not necessary to pass into the <Commandlet>.
* Run <Commandlet> with Project specified so it can load project-specific statistic entries
* Remove -NoEnginePlugins parameter (projects require plugins!)
* Always do many hundreds of frames after we request PIE (vs. a time span which can be taken up mostly with preparing to PIE)
* Remove GC from the automated test function, it was just legacy and it betrays how realistic the test is
* Disable sleep in Editor when it doesn't have focus only in Deterministic mode

#rnx
#jira none
#rb francis.hurteau
#preflight 610cb65b0d783d000149b82f
#preflight 610caadd72fe2c0001a73f02

#ROBOMERGE-SOURCE: CL 17086203 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17086233 by geoff evans in ue5-release-engine-test branch]
2021-08-06 14:49:22 -04:00
david harvey
fe2267f110 verify directory access permission on Windows before expanding wildcards in Turnkey.
#jira none
#rnx
#rb nuno.leiria

#ROBOMERGE-SOURCE: CL 17080840 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)

[CL 17080851 by david harvey in ue5-release-engine-test branch]
2021-08-06 04:51:29 -04:00
william ewen
b647ab0201 Fix device usage comment calls for any UnrealTests that don't have an AppInstance
#rb J.Baumgartner

#ROBOMERGE-OWNER: j.baumgartner
#ROBOMERGE-AUTHOR: william.ewen
#ROBOMERGE-COMMAND: _robomerge[starship] UE5-Main
#ROBOMERGE-SOURCE: CL 17057251 via CL 17059887 via CL 17060149 via CL 17062904
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v853-17066230)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17078316 by william ewen in ue5-release-engine-test branch]
2021-08-05 19:33:44 -04:00
joe kirchoff
d2530f335c Change CheckPerforceCase to print warnings instead of errors
#rb trivial
#rnx

#ROBOMERGE-SOURCE: CL 17042143 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v848-17036448)

[CL 17042428 by joe kirchoff in ue5-release-engine-test branch]
2021-08-03 19:58:07 -04:00
josh engebretson
c882966b35 Gauntlet: Fixing issue with reservation renewal, adding Horde job and step ids to reservations. Cleaning up some legacy workers configuration
#jira UE-120801
#rnx

#ROBOMERGE-SOURCE: CL 17037659 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v848-17036448)

[CL 17037684 by josh engebretson in ue5-release-engine-test branch]
2021-08-03 16:18:05 -04:00
aurel cordonnier
dc4bf61540 Merge from Release-Engine-Staging @ 17030559 to Release-Engine-Test
This represents UE4/Main @ 17030256 and Dev-PerfTest @ 17030553

[CL 17031509 by aurel cordonnier in ue5-release-engine-test branch]
2021-08-03 11:56:47 -04:00
jerome delattre
5a879097ef Copy the ImageMagick native Dlls to Gauntlet bin output
#jira UE-120174
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 17024234 via CL 17029377 via CL 17029521 via CL 17029598
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v846-17029325)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17029628 by jerome delattre in ue5-release-engine-test branch]
2021-08-03 09:30:57 -04:00
jonathan adamczewski
04e2d17afc P4WriteConfig:
Write p4config.txt with a trailing newline.

#jira none
#trivial

#ROBOMERGE-SOURCE: CL 17020839 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v839-17012307)

[CL 17020849 by jonathan adamczewski in ue5-release-engine-test branch]
2021-08-02 17:36:06 -04:00
ben zeigler
e9bd4ff609 #jira UE-120464 Add Custom Config directory feature, which allows specifying an override directory to support shipping multiple types of builds per platform
Setting CustomConfig=Foo in a Target.cs file will cause it to overlay config files from Project/Config/Custom/Foo on top of the defaults, allowing easy override of things like OSS settings to support multiple stores
In development, -CustomConfig=foo can be specified in both C++ and C# to enable the same functionality, which can be used to select between multiple different stage/deploy configurations in a build script
#rb josh.adams, ben.marsh

#ROBOMERGE-SOURCE: CL 17017130 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v839-17012307)

[CL 17017144 by ben zeigler in ue5-release-engine-test branch]
2021-08-02 12:22:46 -04:00
geoff evans
c0d2e3e3c4 IterationProfile: Add -dx11 to side step UE-117842
#jira none
#rb trivial
#rnx
#preflight 6102ecc8b4288d0001e8ae70

#ROBOMERGE-SOURCE: CL 17001835 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v838-16927207)

[CL 17001849 by geoff evans in ue5-release-engine-test branch]
2021-07-29 17:55:13 -04:00