This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971
[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
Local shader compiler:
- Not making 1 job batches (for High prio jobs) on startup
- Randomizing pending job selection to reduce chance that multiple 60+ sec jobs get into a single batch
(the above have the most effect on speed up, below is optional/misc)
- Pushing the completed jobs back to manager before, and not after, pulling new ones to reduce idle time
- Processing jobs in FIFO rather than LIFO order (change to LIFO seems like an ODSC regression? Hard to say definitively if it's a regression from the numbers, but seems odd to have the earliest jobs processed last)
- Parallel processing of input and output files (starts sequential by default to reduce the CPU overhead, but is enabled if we ever see too long write/read)
- More insights scopes
Distributed:
- Not avoiding local machine for XGE on startup
Both:
- Input file compression (disabled by default, need to better figure out when it's beneficial in a general case. Mostly for people with really slow I/O and XGE over VPN).
- More stats, also more dense stat output. Removed an unimportant one, added stats about the batches.
#rb Jason.Nadro, Ben.Ingram
#[review] [at]Jason.Nadro, [at]Ben.Ingram
#preflight 6132ec79bf137d0001ae91ee
#jira UE-125101
#ushell-cherrypick of 17448576 by Arciel.Rekman
#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 17448989 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v870-17433530)
[CL 17448996 by arciel rekman in ue5-release-engine-test branch]
- Increased the number of jobs per XGE worker to reduce the number of times we start them. Since XGE SCWs aren't used for latency-critical jobs, this is expected to not cause user-visible delays.
- Added an ability to detect a hung distributed controller (but the build would be currently broken anyway as the jobs aren't reissued).
- Added a check for XGE system service running to avoid attempting to launch XGE on machines without it.
- Removed the code to even attempt to launch XGE XML interface.
- Also added some more logs about what's happening, and reduced the job cache verbosity.
#rb Jason.Nadro, Ben.Ingram, Danny.Couture
#jira none
#ROBOMERGE-OWNER: Arciel.Rekman
#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 15878601 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v786-15839533)
#ROBOMERGE-CONFLICT from-shelf
[CL 15878696 by Arciel Rekman in ue5-main branch]
- This helps with overall system responsiveness when many shader workers are spawned at the same time
#rb Arciel.Rekman
[CL 15580762 by danny couture in ue5-main branch]
- Avoid directory scanning for .ini files by restoring already initialized config cache state on the workers
- Avoid directory scanning for modules by restoring already initialized modulemanager state on the workers
- Avoid directory scanning for external profilers DLLs by disabling the option in the build config
- 20s -> 319ms of FEngineLoop::PreInit which was caused by directory scan through XGE remote filesystem
- 5% -> 96% efficiency when computing the effective work against process total time for remotely built shaders
- 5m36 ->1m26s to run "recompileshaders all" console command including waiting on async built shaders
#rb Danny.Couture (authored), Luke.Thatcher, Steve.Robb, Josh.Adams
[FYI] Bob.Tellez, Danny.Couture
#ROBOMERGE-OWNER: Arciel.Rekman
#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 11106212 via CL 11106216
#ROBOMERGE-BOT: (v640-11091645)
[CL 11106241 by Arciel Rekman in Main branch]
Add Compiletime stat
Add some tooltips
#rb ben.salem, rolando.caloca
#ROBOMERGE-SOURCE: CL 10808389 via CL 10808392
#ROBOMERGE-BOT: (v610-10636431)
[CL 10808393 by jonas meyer in Main branch]
