[FYI] Zousar.Shaker
Original CL Desc
-----------------------------------------------------------------
Incremental step towards being able to stage to both a pak build as well as a nopak (streamng) build from a snapshot entirely stored in zenserver (no loose files on the filesystem except a ue.projectstore file).
Key changes:
- Removed the use of cookedfiles.manifest
- Changed IoStore mode of UnrealPak to be capable of getting zenserver launch data from either a package store manifest (cbobject, metadata) OR a project store marker file (json)
- Ensured that the UAT and IoStore mode of UnrealPak can launch zenserver reliably by passing along the SponsorProcessId when launching zenserver
- Ensured that shader archives (and their accompanying json metadata files) can be read either as loose files on disk or directly from zenserver if it the data for them is internal to zenserver (ie: they have a valid chunk id)
Remaining work:
- Pak mode of UnrealPak must be able to launch zenserver and pull data from it.
#rb PJ.Kack
[CL 34498668 by zousar shaker in ue5-main branch]
Key changes:
- Removed the use of cookedfiles.manifest
- Changed IoStore mode of UnrealPak to be capable of getting zenserver launch data from either a package store manifest (cbobject, metadata) OR a project store marker file (json)
- Ensured that the UAT and IoStore mode of UnrealPak can launch zenserver reliably by passing along the SponsorProcessId when launching zenserver
- Ensured that shader archives (and their accompanying json metadata files) can be read either as loose files on disk or directly from zenserver if it the data for them is internal to zenserver (ie: they have a valid chunk id)
Remaining work:
- Pak mode of UnrealPak must be able to launch zenserver and pull data from it.
#rb PJ.Kack
[CL 34481636 by zousar shaker in ue5-main branch]
- We could match on irrelevant log line and then be forced to return a partial TSAN report depite not finding our report end delimiter
- We could potentially move the LogCursor one more line than available in the buffer if we end parsing on a line that matches our stack trace pattern since we would call MoveNext() on potentially a null line. This would result in exceptions being thrown while generating events.
[CL 33313709 by kevin macaulayvacher in ue5-main branch]
Doing so the TSAN log matcher can now generate logs that contain the full TSAN error context in a single log. An TSAN IssueHandler has been added which looks for the summary log event and generates an issue keyed off the summary error reason and the file in error. The goal being to generate an issue per unique TSAN error
The TSANLogMatcher has had a test added to ensure we properly log only TSAN errors, and the file was renamed to fix warnings regarding file name and class name must matching.
EditorPerf timeouts have been reduced as we were waiting for 8 hours in cases when no tests were running due to errors in the config
#jira UE-212560
#rb Ben.Marsh
[FYI] Francis.Hurteau
[CL 33261211 by kevin macaulayvacher in ue5-main branch]
-- The "SUMMARY" message is the only log line treated as an error since it is information (description of problem, and file+line to address) but also not overly specific such that we should be able to make JIRA issues from these log lines without generating duplicate issues for the same race from different callsites.
- Changes EditorPerf TSAN job to use -VeryVerbose in UAT so that stdout and stderr are redirected for logging. TSAN emits logs to stderr/stdout (depends on environement variable, stderr by default). The logs can be redirected to a log file via envvar TSAN_OPTIONS and specifying a path, however, Horde will only use whatever is written to stdout/stderr for generating issues, so unless a post process is added to forward the TSAN log output after the application has terminated, having stdout redirected stderr/stdout by default for TSAN jobs is simplest approach.
- Added two new KnownLogEvents `Sanitizer_Thread` and `Sanitizer`. `Sanitizer_Thread` is used by the added matcher, but `Sanitizer` was also added so that we may follow the pattern of letting the first id represent the broader category of events
#rb Bryan.Johnson, Ben.Marsh
#jira UE-207074
[FYI] Francis.Hurteau
[CL 33007156 by kevin macaulayvacher in ue5-main branch]
Skip build flag is passed to UBT so that only target files can be generated without performing a full build operation.
Fixing an issue in the CompileTask where AllowAllCores would always be false
[REVIEW] [at]Joe.Kirchoff [at]Ben.Marsh [at]Ryan.Hummer [at]Gary.Yuan [at]Philip.Cosgrave
#rb Ben.Marsh
[CL 32489167 by eric knapik in ue5-main branch]
