Commit Graph

27 Commits

Author SHA1 Message Date
aurel cordonnier
d17d20ca36 Merge from Release-Engine-Test @ 16758890 to UE5/Main
This represents UE4/Main @ 16738161 and Dev-PerfTest @ 16737719 (and Release-17.00 @ 16658211)

[CL 16763350 by aurel cordonnier in ue5-main branch]
2021-06-23 17:51:32 -04:00
aurel cordonnier
e0ad4e25df Merge from Release-Engine-Test @ 16624776 to UE5/Main
This represents UE4/Main @ 16579691 and Dev-PerfTest @ 16579576

[CL 16625248 by aurel cordonnier in ue5-main branch]
2021-06-10 13:13:24 -04:00
Jerome Delattre
c7dc6ed851 Gauntlet - Add support for Unit and Baseline to Telemetry data
#jira none
#rb none
#rnx

[CL 16496261 by Jerome Delattre in ue5-main branch]
2021-05-27 21:57:49 -04:00
aurel cordonnier
50944fd712 Merge UE5/RES @ 16162155 to UE5/Main
This represents UE4/Main @ 16130047 and Dev-PerfTest @ 16126156

[CL 16163576 by aurel cordonnier in ue5-main branch]
2021-04-29 19:32:06 -04:00
ben salem
472299ebcf Add InsufficientDevices as a specific TestResult type for Gauntlet.
#ROBOMERGE-OWNER: Ben.Woodhouse
#ROBOMERGE-AUTHOR: ben.salem
#ROBOMERGE-COMMAND: _robomerge ue5-main
#ROBOMERGE-SOURCE: CL 16037939 via CL 16038037 via CL 16040838 via CL 16040843 via CL 16040847 via CL 16055143
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Main) (v789-15992632)

[CL 16115258 by ben salem in ue5-main branch]
2021-04-26 12:16:15 -04:00
Jerome Delattre
6c6bcf0814 Gauntlet - Add build command to publish telemetry from CSV
+ avoid using ITestContext instead use a blackboard type object as context to store abitrary data

at the command line:
 Engine\Build\BatchFiles\RunUAT.bat PublishUnrealAutomationTelemetry -CSVFile=C:\temp\myfile.csv -project=Reverb -changelist=111 -branch=ue5-main

in buildgraph:
<Command Name="PublishUnrealAutomationTelemetry" Arguments="-CSVFile=C:\temp\myfile.csv -project=Reverb -changelist=111 -branch=ue5-main" />

Support csv re-mapping:
 -CSVMapping=DataPoint:SomeName,TestName:SomeKey

#jira none
#rnx
#rb Geoff.Evans

[CL 16056211 by Jerome Delattre in ue5-main branch]
2021-04-19 18:50:12 -04:00
eric mcdaniel
d846a46845 Additional workarounds for explicit references to optional console platforms in title build system
- includes a cherrypick of 15851870 by jeff.newquist as well as additional fixes
- some changes here are temporary and will be revisited via FORT-363688

#rb David.Harvey, Andrew.Grant, Luke.Thatcher
#jira none

[CL 15976966 by eric mcdaniel in ue5-main branch]
2021-04-12 09:11:22 -04:00
Marc Audy
0cbbc781ca Merge UE5/Release-Engine-Staging @ 15740152 to UE5/Main
This represents UE4/Main @ 15709114

[CL 15740605 by Marc Audy in ue5-main branch]
2021-03-18 15:20:03 -04:00
Jerome Delattre
886c5c2c83 Gauntlet - Add Context property to TelemetryData
#jira
#rnx
#rb none

[CL 15654353 by Jerome Delattre in ue5-main branch]
2021-03-09 10:50:56 -04:00
Jerome Delattre
4d77c4f788 Gauntlet - Formalize telemetry support in Gauntlet
- add simple data structure to hold the data
- introduce database config and driver
- support sending telemetry data from UnrealTestNode

#jira UEENGQA-38048
#rb Andrew.Grant,Bob.Ferriera

[CL 15480886 by Jerome Delattre in ue5-main branch]
2021-02-19 20:18:31 -04:00
Marc Audy
9753392e2b Merge UE5/RES CL# 15462083 to UE5/Main
This represents UE4/Main @ 15414221

[CL 15463811 by Marc Audy in ue5-main branch]
2021-02-18 18:13:28 -04:00
Marc Audy
bc88b73a29 Merge Release-Engine-Staging to Main @ CL# 15151250
Represents UE4/Main @ 15133763

[CL 15158774 by Marc Audy in ue5-main branch]
2021-01-21 16:22:06 -04:00
Jerome Delattre
bb9343019d Make sure path are fully qualified when coping artifact files with Gauntlet
#jira none
#rnx
#rb none

[CL 14874002 by Jerome Delattre in ue5-main branch]
2020-12-07 17:48:26 -04:00
Jerome Delattre
ee8b6387d8 # Simplify gauntlet report for Horde
Introduce Gauntlet.ITestReport interface
 - enable attaching artifact to report through ITestReport.AttachArtifact(string)
 - enable reporting errors and warnings to ITestReport

Deprecate CreateReport(TestResult, Context, Build, Artifacts, ArtifactPath) and favor to CreateReport(ITestReport)

Generate a simplified report for Horde Test Data

Better support copying artifacts with long path on Windows

#jira none
#rb Andrew.Grant

[CL 14859929 by Jerome Delattre in ue5-main branch]
2020-12-04 17:30:52 -04:00
Jerome Delattre
4d865f6c0b Editgrate 14600524
Avoid redundant error report in the log by implementing a new property LogWarningsAndErrorsAfterSummary to ITestNode

#jira none
#rnx
#rb josh.engebretson, andrew.grant

[CL 14809680 by Jerome Delattre in ue5-main branch]
2020-11-24 11:10:24 -04:00
Marc Audy
7379fa99c5 Merging //UE5/Release-Engine-Staging to Main (//UE5/Main) @ 14229157
[CL 14233282 by Marc Audy in ue5-main branch]
2020-09-01 14:07:48 -04:00
ben salem
ae0820e1f4 Add support for some basic documentation being able to be built into gauntlet nodes. Includes support for describing commandline arguments as well as sample commandlines.
#rb clayton.langford

#ROBOMERGE-SOURCE: CL 12646231 via CL 12646246 via CL 12649255 via CL 12649452 via CL 12649499 via CL 12649616
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v675-12543919)

[CL 12649737 by ben salem in Main branch]
2020-04-07 00:50:43 -04:00
Ryan Durand
9ef3748747 Updating copyrights for Engine Programs.
#rnx
#rb none
#jira none

#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869242 in //Fortnite/Release-12.00/... via CL 10869536
#ROBOMERGE-BOT: FORTNITE (Main -> Dev-EngineMerge) (v613-10869866)

[CL 10870955 by Ryan Durand in Main branch]
2019-12-26 23:01:54 -05:00
william ewen
8d83ff0cca Gauntlet: Adding "NotBulk" flag, so we can specifically exclude bulk builds if we wish to
#rb Andrew.Grant
#rnx


#ROBOMERGE-SOURCE: CL 7651008 via CL 7654234
#ROBOMERGE-BOT: (v372-7473910)

[CL 7654408 by william ewen in Main branch]
2019-07-29 17:39:09 -04:00
Ben Marsh
0cc6e3dca6 Copying //UE4/Dev-Build to Dev-Main (//UE4/Dev-Main)
#rb none
#rnx

[CL 6631504 by Ben Marsh in Main branch]
2019-05-24 11:51:54 -04:00
andrew grant
91c518c863 Gauntlet -
ITestNode now defines GetWarnings and GetErrors calls for tests to return instances that occurred during a test.

Split creation of Summary report in UnrealTestNode into Header/Body functions to allow tests to more easily customize these.

Added GetLogChannels() function to UnrealLogParser that will return all lines from multiple channels

Added AutomatonLogParser that specifically parses the output of automation (non-Gauntlet) tests.

#rb CRd
[at]josh.engebretson [at]clayton.langford [at]ben.salem


#ROBOMERGE-SOURCE: CL 6241133 via CL 6246005

[CL 6246055 by andrew grant in Main branch]
2019-05-02 00:10:12 -04:00
clayton langford
2fd5d814b0 Updated blacklist config directory location.
#rnx


#ROBOMERGE-SOURCE: CL 5588255 via CL 5588260 via CL 5588262 via CL 5588263 via CL 5589115

[CL 5589118 by clayton langford in Main branch]
2019-03-27 10:37:54 -04:00
Ben Marsh
530369c613 Merging //UE4/Dev-Main to Dev-Build (//UE4/Dev-Build)
#rb none
#rnx

[CL 4662695 by Ben Marsh in Dev-Build branch]
2018-12-14 14:49:12 -05:00
Ben Marsh
7598af0532 Update copyright notices to 2019.
#rb none
#lockdown Nick.Penwarden

[CL 4662404 by Ben Marsh in Main branch]
2018-12-14 13:41:00 -05:00
Ben Marsh
b0da1f4fe7 Merging //UE4/Dev-Main to Dev-Build (//UE4/Dev-Build)
#rb none

[CL 4577636 by Ben Marsh in Dev-Build branch]
2018-11-19 10:12:17 -05:00