Commit Graph

2232 Commits

Author SHA1 Message Date
kevin macaulayvacher
abfde6cf9f Remove unnecessary using directives from threadsanitizerissuehandler. Fixing issue=460206
[CL 33262522 by kevin macaulayvacher in ue5-main branch]
2024-04-26 09:50:58 -04:00
kevin macaulayvacher
8b887d6c5c Adds a new step to TSAN error process to allow a report of only the unique TSAN errors to be displayed. This step parses the output log from the test run and displays the unique errors and removes any interwoven log lines that are irrelevant.
Doing so the TSAN log matcher can now generate logs that contain the full TSAN error context in a single log. An TSAN IssueHandler has been added which looks for the summary log event and generates an issue keyed off the summary error reason and the file in error. The goal being to generate an issue per unique TSAN error

The TSANLogMatcher has had a test added to ensure we properly log only TSAN errors, and the file was renamed to fix warnings regarding file name and class name must matching.

EditorPerf timeouts have been reduced as we were waiting for 8 hours in cases when no tests were running due to errors in the config

#jira UE-212560
#rb Ben.Marsh
[FYI] Francis.Hurteau

[CL 33261211 by kevin macaulayvacher in ue5-main branch]
2024-04-26 08:42:40 -04:00
jerome delattre
bc3b95d1e6 Improve Gauntlet issue handling by adding stream id and node name to generate key as long as a Callstack property is not found.
#jira UE-208397
#rnx
#rb Ben.Marsh, sebastian.lewicki

[CL 33243600 by jerome delattre in ue5-main branch]
2024-04-25 16:30:45 -04:00
ben marsh
bf8485156c Horde: Do not return a default localhost server URL if nothing is configured.
[CL 33235187 by ben marsh in ue5-main branch]
2024-04-25 12:39:17 -04:00
henrik karlsson
7fc8ad83e8 [UBA]
* Integrated uba cache client in to UBT. This is the first iteration and is only really tested on UnrealEditor win64 development -nopch.

Test by starting a UbaCacheService.exe somewhere and then compile using -UbaCache=<thatmachine> -UbaWriteCache

#rb joe.kirchoff

[CL 33214931 by henrik karlsson in ue5-main branch]
2024-04-24 17:58:07 -04:00
ben marsh
f0e5e43b14 EpicGames.Perforce: Fix errors parsing change views from labels.
#jira UE-213169

[CL 33207702 by ben marsh in ue5-main branch]
2024-04-24 14:41:10 -04:00
henrik karlsson
026b657d49 [UBT]
* Fixed so Classes/Public/Internal/Private includes are only added if there are files directly or indirectly under directory. Change is to make rsp files deterministic even if p4 left empty directories behind

#rb joe.kirchoff

[CL 33206443 by henrik karlsson in ue5-main branch]
2024-04-24 14:08:39 -04:00
ben marsh
48f508b41a Horde: Use the shared registry location to configure the Horde command line tool on Windows, and use a .horde.json in the user folder on Mac and Linux.
[CL 33205896 by ben marsh in ue5-main branch]
2024-04-24 13:51:58 -04:00
ben marsh
af2b867a08 Horde: Rename step -> job step, report -> job report, batch -> job batch in job responses.
[CL 33191561 by ben marsh in ue5-main branch]
2024-04-23 22:34:38 -04:00
ben marsh
a3aae9d5ea Horde: Tweak order of properties in job responses.
#rnx

[CL 33191293 by ben marsh in ue5-main branch]
2024-04-23 22:07:08 -04:00
ben marsh
c545131652 Horde: Include graph information in job responses.
[FYI] Josh.Engebretson
#jira UE-212016

[CL 33189374 by ben marsh in ue5-main branch]
2024-04-23 20:50:49 -04:00
joe kirchoff
53d9e6fef5 DirectoryItem.ContainsFiles
#rnx

[CL 33183766 by joe kirchoff in ue5-main branch]
2024-04-23 18:54:58 -04:00
ben marsh
2fe958bd2a Horde: Rename uses of "LogFile" to "Log". We started renaming all these things some time ago (eg. LogId) but didn't finish it.
[CL 33115838 by ben marsh in ue5-main branch]
2024-04-19 16:36:22 -04:00
ben marsh
7e14d9c615 Horde: Add a service to expire jobs after a period of time.
Jobs are kept forever by default, though this may be modified through the ExpireAfterDays parameter in the JobOptions object in streams or projects.

#jira UE-212550

[CL 33109374 by ben marsh in ue5-main branch]
2024-04-19 13:44:21 -04:00
carl bystrom
784bc1dae8 Horde: Add auto conform for agents with workspaces below a certain free disk space threshold
#jira UE-194184
[FYI] Ryan.Hummer, Bryan.Johnson

[CL 33102678 by carl bystrom in ue5-main branch]
2024-04-19 10:44:11 -04:00
carl bystrom
0bb97b728b Horde: Move KnownPropertyNames to EpicGames.Horde
Both agent and server are making use of these.

#rnx

[CL 33099296 by carl bystrom in ue5-main branch]
2024-04-19 07:25:27 -04:00
carl bystrom
6284e919f6 Horde: Add min scratch space to workspace config
#jira UE-194184

[CL 33098372 by carl bystrom in ue5-main branch]
2024-04-19 05:42:46 -04:00
joshua shlemmer
acedfad23c .NET static analysis warnings in LogMessages.cs
[CL 33085216 by joshua shlemmer in ue5-main branch]
2024-04-18 16:53:40 -04:00
joshua shlemmer
dceaf4ca2f Hordehttpclient changes adding GetGraph and log search calls.
#rb gary.yuan

[CL 33080726 by joshua shlemmer in ue5-main branch]
2024-04-18 14:50:59 -04:00
ben marsh
144fc2b4bd Horde: Prevent hypens before a period in a sanitized string id.
[CL 33044625 by ben marsh in ue5-main branch]
2024-04-17 15:07:49 -04:00
ben marsh
768649e763 EpicGames.Perforce: Fix parsing of records with multiple arrays. All properties for the entire tree of possible records for a response are looked up by tag before identifying the target object via the array index, rather then recursively trying to parse complete records. It is possible for one record to have multiple child records, and also possible that tags for child records are serialized out of order.
One regression with this approach is that the required option is not validated for subelement types; since this is more of a validation of assumptions of the data format rather than an expected error case, it should be fine.

[CL 33038497 by ben marsh in ue5-main branch]
2024-04-17 12:03:47 -04:00
ben marsh
2dd301febe Create an AutomationTools.Tests project, and move tests from Horde.Agent into it. Included in UE5 default solution for better visibility.
[FYI] carl.bystrom, kevin.macaulayvacher

[CL 33036388 by ben marsh in ue5-main branch]
2024-04-17 10:39:36 -04:00
ben marsh
c1021f6bf8 Horde: Handle zero byte reads due to socket shutdown when reading messages into a compute buffer.
[CL 33034491 by ben marsh in ue5-main branch]
2024-04-17 09:31:45 -04:00
kevin macaulayvacher
1c564f5450 - Adds a ThreadSanitizer Structured Logging Matcher so we may contextualize TSAN output in Horde logs. It specifically will attempt to annotate files, symbols, and symbol addresses in callstacks, warning on TSAN "WARNING" messages and "SUMMARY" message.
-- The "SUMMARY" message is the only log line treated as an error since it is information (description of problem, and file+line to address) but also not overly specific such that we should be able to make JIRA issues from these log lines without generating duplicate issues for the same race from different callsites.
- Changes EditorPerf TSAN job to use -VeryVerbose in UAT so that stdout and stderr are redirected for logging. TSAN emits logs to stderr/stdout (depends on environement variable, stderr by default). The logs can be redirected to a log file via envvar TSAN_OPTIONS and specifying a path, however, Horde will only use whatever is written to stdout/stderr for generating issues, so unless a post process is added to forward the TSAN log output after the application has terminated, having stdout redirected stderr/stdout by default for TSAN jobs is simplest approach.
- Added two new KnownLogEvents `Sanitizer_Thread` and `Sanitizer`. `Sanitizer_Thread` is used by the added matcher, but `Sanitizer` was also added so that we may follow the pattern of letting the first id represent the broader category of events

#rb Bryan.Johnson, Ben.Marsh
#jira UE-207074
[FYI] Francis.Hurteau

[CL 33007156 by kevin macaulayvacher in ue5-main branch]
2024-04-16 13:44:37 -04:00
ben marsh
1db0da441d BuildGraph: Allow specifying an IgnoreModified flag on node attributes to ignore mismatched timestamps/lengths of files matching a particular pattern (accepts p4-style wildcards and multiple rules separated by semicolons).
[FYI] Eric.Knapik

[CL 32997469 by ben marsh in ue5-main branch]
2024-04-16 07:46:44 -04:00