Commit Graph

18657 Commits

Author SHA1 Message Date
Josh Engebretson
45843b9fcc Horde: Add step retries to job operations bar
#jira none
#fyi ryan.hummer
#rnx
#preflight none
#skipci

[CL 25333613 by Josh Engebretson in ue5-main branch]
2023-05-04 09:09:23 -04:00
Josh Engebretson
6cafa143da Horde: Fix meta data collection index
#jira none
#rnx
#preflight none

[CL 25333231 by Josh Engebretson in ue5-main branch]
2023-05-04 08:34:46 -04:00
thomas ross
87ff95b9a9 (Manual Merge of CL 25330963) VirtualFileSystem Cosmetic File Streaming Updates:
- Generate new predownloaded cosmetic bundles FortniteBROnDemand and FortniteBROnDemandOptional. These bundles store a copy of all of the ondemand files so that users can opt-out of downloading on demand.
- Implemented new user settings to disable cosmetic streaming on Android and just pre-download all cosmetics
- New settings fallback to disable cosmetic streaming automatically if predownloaded cosmetic data is already on disk. This supports both Android Bulk builds and PC where the launcher installs this data.
- Also disable CosmeticStreaming when the predownloaded cosmetic content is installed (which is requested automatically if the user setting is flipped)
- Implemented new FortCosmeticStreamingModule modular feature to remove some hard dependencies and make code cleaner.
#rb Swarm
#tests Android Dev Client, Windows Dev Client

[CL 25331176 by thomas ross in ue5-main branch]
2023-05-04 00:16:13 -04:00
Ben Marsh
bb514c607f Horde: Create lease documents for compute jobs, allowing them to be cancelled without 404'ing.
#preflight none

[CL 25331123 by Ben Marsh in ue5-main branch]
2023-05-03 23:51:59 -04:00
Ben Marsh
0ede5eada2 UBT: Match engine-style script errors in logs.
#preflight none

[CL 25330216 by Ben Marsh in ue5-main branch]
2023-05-03 21:04:34 -04:00
Joe Kirchoff
99c6aec40c AutomationTool: Fix condition in .csproj (via UDN)
#rnx
#rb trivial
#preflight 6452f13f6538e45f75e937d2

[CL 25329475 by Joe Kirchoff in ue5-main branch]
2023-05-03 20:02:42 -04:00
Yuriy ODonnell
bb51087f42 unsync - Finish up --scavenge mode implementation, bump version to 1.0.51
* Count scavenged blocks as "base" data for stats/telemetry
* Gracefully handle missing/invalid scavenge root path
* Cancel reading blocks from a particular source if a hash mismatch was found

#rb none
#jira UE-178864
#preflight skip

[CL 25326911 by Yuriy ODonnell in ue5-main branch]
2023-05-03 17:28:38 -04:00
Ben Marsh
1f96a6852a Horde: Make step and batch ids stable across calls to CreateOrUpdateBatches().
#preflight none

[CL 25325740 by Ben Marsh in ue5-main branch]
2023-05-03 16:48:31 -04:00
Ben Marsh
f5ec1940c3 Horde: Return the correct id of a new retried node in the response.
#preflight none

[CL 25324368 by Ben Marsh in ue5-main branch]
2023-05-03 16:00:25 -04:00
Ben Marsh
5c39840637 Horde: Retry upstream steps if a skipped node is flagged for retry.
#preflight none
#jira UE-132884

[CL 25322132 by Ben Marsh in ue5-main branch]
2023-05-03 14:54:25 -04:00
Josh Engebretson
0358122aae Horde: More markdown rendering tweaks
#jira none
#rnx
#preflight none
#skipci

[CL 25319758 by Josh Engebretson in ue5-main branch]
2023-05-03 13:11:17 -04:00
Josh Engebretson
3f7fcb8501 Horde Docs view tweaks
#jira UE-184961
#preflight none
#skipci
#rnx

[CL 25319303 by Josh Engebretson in ue5-main branch]
2023-05-03 12:50:09 -04:00
Ben Marsh
bb06ec7232 Horde: When retrying a job step, also retry all its dependencies.
#preflight none

[CL 25319192 by Ben Marsh in ue5-main branch]
2023-05-03 12:44:54 -04:00
zack neyland
ffb0c475e5 UBT: Remove explicit objC ABI flags.
#jira UE-113905
#preflight na

[CL 25318459 by zack neyland in ue5-main branch]
2023-05-03 12:05:46 -04:00
carl bystrom
90c05c5c58 Horde: Log unhandled event types for ClickHouseTelemetrySink
Also lower the metric sending interval from Horde server to ClickHouse.

#preflight none

[CL 25318438 by carl bystrom in ue5-main branch]
2023-05-03 12:04:58 -04:00
dan elksnitis
b52faed5d5 [shaders]
- add new IShaderFormat API for separate preprocessing and compilation; backends can implement one or the other depending on the return value of SupportsIndependentPreprocessing
- add support for executing preprocessing in the cook process prior to job submission and constructing job input hashes based on preprocessed source (and a subset of the environment used as compile inputs). controlled by a cvar for now and disabled by default
- add a BaseShaderFormat class in ShaderCompilerCommon which implements common behaviour for output of debug data - note this function is only called for formats which support independent preprocessing, so is expected to be used only by formats which have been converted to use this API
- add new cvars for output of some additional shader debug data - 1. a txt file containing the input hash a.k.a. job cache key 2. a text file containing all diagnostic messages (errors and warnings) for the job
- minor change to how input hashes are constructed for pipeline jobs - sum hashes as 256-bit ints instead of adding to a buffer and re-hashing. faster and simpler, and also more collision resistant (sum of two well distributed hashes equally well distributed)

#rb Jason.Nadro
#rb Yuriy.ODonnell
#preflight 64512c88c86798f650b953d3

[CL 25317218 by dan elksnitis in ue5-main branch]
2023-05-03 10:17:48 -04:00
carl bystrom
ae916af007 Horde: Replace DogStatsD with OpenTelemetry metrics
#preflight 645248a00e33f2d51dad6a25

[CL 25316072 by carl bystrom in ue5-main branch]
2023-05-03 07:47:47 -04:00
carl bystrom
3cc2450a51 Horde: Remove OpenTracing use coming from Datadog and Serilog NuGet packages
#preflight 64522b76d863ba26210ac09b

[CL 25315505 by carl bystrom in ue5-main branch]
2023-05-03 06:02:50 -04:00
David Harvey
d36f6a11f1 Fix for per-project packaging images.
#rnx
#jira UE-17918
#rb trivial
#preflight skip

[CL 25315169 by David Harvey in ue5-main branch]
2023-05-03 04:34:16 -04:00
Ben Marsh
82ae935ede Horde: Return flags indicating whether various admin-related features should be enabled in the dashboard for the current user.
#preflight none
#fyi Josh.Engebretson

[CL 25312324 by Ben Marsh in ue5-main branch]
2023-05-02 23:22:51 -04:00
Ben Marsh
d31ae5f9cd Horde: Add custom ACLs for modifying notices.
#preflight none

[CL 25312245 by Ben Marsh in ue5-main branch]
2023-05-02 23:00:52 -04:00
Ben Marsh
3dfb0cdbba Horde: Run the Mongo upgrade service from non-worker nodes.
Some collections are only instantiated when required for a controller; this may not happen on worker nodes.

#preflight none

[CL 25312053 by Ben Marsh in ue5-main branch]
2023-05-02 22:20:33 -04:00
Ben Marsh
cbf8d0dc15 Horde: More logging for index updates.
#preflight none

[CL 25312021 by Ben Marsh in ue5-main branch]
2023-05-02 22:05:58 -04:00
Ben Marsh
2f8663fb6e Horde: More logging for MongoDB upgrades.
#preflight none

[CL 25311954 by Ben Marsh in ue5-main branch]
2023-05-02 21:41:50 -04:00
Ben Marsh
c94168b27b Horde: Add log output for when we can't acquire the upgrade schema lock.
#preflight none

[CL 25311841 by Ben Marsh in ue5-main branch]
2023-05-02 21:11:47 -04:00