Commit Graph

67 Commits

Author SHA1 Message Date
Ryan Hummer
26a78aed98 Adding temp storage timestamp exclusion for tbbmalloc.psym
#rnx
#jira UE-185067
#rb none
#preflight none

[CL 25334983 by Ryan Hummer in ue5-main branch]
2023-05-04 10:37:21 -04:00
Ben Marsh
868e20b281 Add temp storage timestamp exclusion for tbbmalloc.pdb.
#preflight none

[CL 25284039 by Ben Marsh in ue5-main branch]
2023-05-01 10:57:11 -04:00
Ben Marsh
d32e8436c1 BuildGraph: Store artifact keys in the tag manifest rather than in the exported graph definition, since they may include non-constant values from build arguments.
#preflight none

[CL 25057984 by Ben Marsh in ue5-main branch]
2023-04-14 21:42:00 -04:00
carl bystrom
c1d1bd7cdf BuildGraph: Use .NET API for copying temp storage dirs and files when running under Wine
#preflight 641083dd3c2db40029dc5983

[CL 24635634 by carl bystrom in ue5-main branch]
2023-03-14 10:46:03 -04:00
Ben Marsh
e141cb6196 More UAT structured logging fixes.
#preflight 6408dee3b0544ef0b4bca696

[CL 24564590 by Ben Marsh in ue5-main branch]
2023-03-08 14:32:15 -05:00
Ben Marsh
235eb77726 Change to structured logging in other AutomationTool projects.
#preflight 6408c4478c0039bbf750e861

[CL 24562553 by Ben Marsh in ue5-main branch]
2023-03-08 12:43:35 -05:00
Ben Marsh
89d6613573 UAT: Refactor AutomationUtils and BuildGraph to use structured logging.
#preflight 6407c4c26e1dadfcdab25849

[CL 24553706 by Ben Marsh in ue5-main branch]
2023-03-07 21:23:47 -05:00
Ben Marsh
d71324777d BuildGraph: Fix temp storage copy actions running 5 times, even if it succeeds on the first attempt.
#fyi Chad.Garyet
#preflight none

[CL 22353978 by Ben Marsh in ue5-main branch]
2022-10-05 09:44:17 -04:00
brandon schaefer
0f99d8453f Workaround for tbb.psym colliding from multiple clients copying tbb.psym
#jira UE-161242
#rb none
#preflight none

[CL 21531151 by brandon schaefer in ue5-main branch]
2022-08-23 22:18:21 -04:00
Nick Darnell
d75cbc192e Core - Updating the TBB copy location so that the dll and pdb end up in the same place to make generating portable symbols easier.
Build - updating the tempstorage build graph script to ignore additonal pdb build products for tbb.

#jira UE-159742
#preflight 62d5e73ec0180d80715a8b8d

[CL 21161802 by Nick Darnell in ue5-main branch]
2022-07-19 10:00:06 -04:00
Ryan Hummer
dcb81492ef Use robocopy/rsync to copy temp storage manifests locally before unzipping them, and use them to copy manifests off to the share at the end of a task.
improves platform reliability, m1 macs struggle to update file attributes when copying to an smb share. rsync ensures the attributes are correctly copied.

#rnx
#jira none
#rb none
#preflight 62d170c93b057e72748b9aa8
#author ryan.durand

[CL 21110668 by Ryan Hummer in ue5-main branch]
2022-07-15 11:15:30 -04:00
ryan hummer
f91cb75ed7 Add retrying to potential network share failure points in temp storage.
#rnx
#jira UE-156108, UE-158088
#rb ryan.hummer
#preflight 62bdeccded35ee71a70ee5a5

#ushell-cherrypick of 20574535 by Ryan.Durand

#ROBOMERGE-AUTHOR: ryan.hummer
#ROBOMERGE-SOURCE: CL 20901486 in //UE5/Release-5.0/... via CL 20902831
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v971-20777995)

[CL 20905677 by ryan hummer in ue5-main branch]
2022-06-30 18:10:24 -04:00
Ryan Hummer
8a3fcafa63 Fix TempStorage Manifest XML Writing
dotnet6 xml serialization nolonger emits new lines, change to using XMLWriter to enable newlines

#rnx
#jira none
#rb ryan.durand
#preflight 62a34bd10251cacef0b2a9e1

[CL 20595487 by Ryan Hummer in ue5-main branch]
2022-06-10 10:45:41 -04:00
Ryan Durand
bf54f10db3 Add retrying to potential network share failure points in temp storage.
#rnx
#jira UE-156108
#rb ryan.hummer
#preflight

[CL 20574535 by Ryan Durand in ue5-main branch]
2022-06-09 08:55:18 -04:00
Ryan Durand
c2a32ddc91 Couple of temp storage clean up changes to let multiple clean up jobs run less painfully.
* Flag folders as delete in progress to allow other clean up jobs to skip a folder and go to the next one.
* Delete from newest to oldest in an attempt to provide a buffer before multiple jobs will attempt to delete the same folder.

#rnx
#jira UE-150924
#rb ryan.hummer,bryan.johnson
#preflight

[CL 20047320 by Ryan Durand in ue5-main branch]
2022-05-04 15:44:17 -04:00
matt johnson
25c86ecac1 UBT: recognize shared libraries with version number suffixes as allowed duplicate build products
Shared libraries on Linux commonly use a scheme of version number suffixes
and/or symlinks to help ensure that client applications can link against the
appropriate version of a library (e.g. "libfoo.so.1.70.0"). This change updates
IsDuplicateBuildProduct() with an additional case to cover these libraries so
that they are not flagged as an error by the build system if multiple steps
produce them.

#jira UE-148232
#rb brandon.schaefer
#preflight 624c76a8712e7c09a6a15d69

[CL 19627733 by matt johnson in ue5-main branch]
2022-04-05 13:17:02 -04:00
Ryan Hummer
e4945b7f53 Clarify error message when scanning temp storage for cleanup
#rnx
#jira none
#rb none
#preflight skip

[CL 19579046 by Ryan Hummer in ue5-main branch]
2022-03-31 17:23:08 -04:00
Ryan Durand
7e494a86dd * Fix spelling on Retrieve
* Copy manifest ZIP files locally before unzipping them.
* Use retries when copying to harden against network blips.

#rnx
#jira UE-139648
#rb ryan.hummer, ben.marsh
#preflight

[CL 19480296 by Ryan Durand in ue5-main branch]
2022-03-23 12:02:11 -04:00
ben marsh
66f37c2bb9 BuildGraph: Take the digest of any *.version files when copying files to temp storage, so their timestamps can be updated without making any content changes.
#preflight none
#jira

#ROBOMERGE-AUTHOR: ben.marsh
#ROBOMERGE-SOURCE: CL 18588704 in //UE5/Release-5.0/... via CL 18588716 via CL 18588728
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v899-18417669)

[CL 18588738 by ben marsh in ue5-main branch]
2022-01-12 15:24:13 -05:00
ryan hummer
c0419eb70e Fixing logging format call to include the file name
#rnx
#jira none

#ROBOMERGE-AUTHOR: ryan.hummer
#ROBOMERGE-SOURCE: CL 18444558 in //UE5/Release-5.0/... via CL 18445395
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18445546 by ryan hummer in ue5-release-engine-test branch]
2021-12-13 13:45:41 -05:00
josh engebretson
ab07ba85cb Add error logging for temp storage zip operations
#jira none
#rnx

#ROBOMERGE-AUTHOR: josh.engebretson
#ROBOMERGE-SOURCE: CL 18421529 in //UE5/Release-5.0/... via CL 18422973
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18423279 by josh engebretson in ue5-release-engine-test branch]
2021-12-09 15:01:49 -05:00
mark lintott
9362edf2fe Catch IO issues when creating or copying ZIP files
#rb josh.engebretson
#jira none

#ROBOMERGE-AUTHOR: mark.lintott
#ROBOMERGE-SOURCE: CL 18420781 in //UE5/Release-5.0/... via CL 18422611
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18422889 by mark lintott in ue5-release-engine-test branch]
2021-12-09 14:52:06 -05:00
ryan durand
a954aa1ab3 Add an explicit error message when we run out of retries and fail to unzip temp storage.
#rnx
#rb none
#jira UE-135258

#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 18243011 in //UE5/Release-5.0/... via CL 18243061
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18243291 by ryan durand in ue5-release-engine-test branch]
2021-11-18 15:38:54 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00
aurel cordonnier
a12d56ff31 Merge from Release-Engine-Staging @ 17791557 to Release-Engine-Test
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485

[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-12 21:21:22 -04:00