Commit Graph

593 Commits

Author SHA1 Message Date
Matt Peters
eb86c46cef Cooker PumpPollables: Add support for manually triggered pollables. Change Pollables into an indirect array, to allow the pointer be used as a handle for the manual trigger. Change pollables into a heap so we don't have to view the entire array each tick.
#rb Devin.Doucette
#rnx
#preflight 62954dcbfb0fca7e581f7872

[CL 20433687 by Matt Peters in ue5-main branch]
2022-05-30 21:44:05 -04:00
Matt Peters
81a3ae5b81 Cooker: Multiprocess support: Remove some deprecated code and move some out of place functions that should not be executed on CookWorker into the proper calling function that will be called on the director only.
#rb Zousar.Shaker
#rnx
#preflight 629135e11b686da53f41b463

[CL 20398652 by Matt Peters in ue5-main branch]
2022-05-27 16:48:59 -04:00
Matt Peters
c21db29e38 Cooker Stats: Fix the "Packages Cooked" stat line to use the number of packages cooked rather than the number of packages multiplied by the number of platforms.
#rb Devin.Doucette
#rnx
#preflight 628d0348c97e4beea3a51802

[CL 20350856 by Matt Peters in ue5-main branch]
2022-05-24 13:04:27 -04:00
Matt Peters
ed6457f264 ITargetPlatform: Delete RefreshSettings. All overrides were removed previously and it is now an empty virtual function and can be removed.
#rb Josh.Adams
#rnx
#preflight 628bd4e1573a7de2c436e39e

[CL 20347492 by Matt Peters in ue5-main branch]
2022-05-24 09:38:04 -04:00
CarlMagnus Nordin
0282a5aef6 Unified Zen and legacy COTF network protocols.
*Reduces code duplication
*Enables the same ODSC flow to be used for both COTF variants
*The client will now autodetect if it should run in Zen mode or not
#rb pj.kack,per.larsson
#preflight 628c79bdf057b981ca479b3e

[CL 20344832 by CarlMagnus Nordin in ue5-main branch]
2022-05-24 02:50:39 -04:00
Catalin Dragoiu
0b478d5694 Cooking Profiler Prototype
#rb Ionut.Matasaru  Matt Peters
#preflight 628b8690693c5e1de2773a39

[CL 20327158 by Catalin Dragoiu in ue5-main branch]
2022-05-23 10:13:54 -04:00
robert manuszewski
f8a812a32f Converting hardcoded short class/enum names to pathnames ahead of ANY_PACKAGE removal
#rb trivial
#jira UE-99463
#preflight 6288fd998828ea88c8aef3d0

#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 20314896 via CL 20314897 via CL 20314903 via CL 20314904
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v948-20297126)

[CL 20315214 by robert manuszewski in ue5-main branch]
2022-05-22 10:30:02 -04:00
Matt Peters
5e4fdf3eaf CookOnTheFlyServer: Combine duplicated variables from cook modes to reduce complexity, avoid the need to check for null, and prepare for the addition of a CookWorker mode.
#rb Zousar.Shaker
#rnx
#fyi CarlMagnus.Nordin, PJ.Kack
#preflight 6287f0582c34da686af615eb

[CL 20301986 by Matt Peters in ue5-main branch]
2022-05-20 16:12:27 -04:00
Matt Peters
593f8117da SavePackage: Undo the call to IsCachedCookedPlatformDataLoaded, since it has caused multiple fallout issues.
Cooker: Fix the issue that we were trying to fix with that previous change; it is specific to WorldPartition and CookPackageSplitter; we need to requery the list of objects in the package that need BeginCacheForCookedPlatformData after we call on WorldPartition to move all of its OFPA objects into the Level object for cooking.
#jira UE-152960
#jira FORT-480046
#jira UE-152856
#rb Zousar.Shaker
#rnx
#preflight 6286a2e8614041edb7235cab

[CL 20286770 by Matt Peters in ue5-main branch]
2022-05-19 16:16:15 -04:00
matt peters
d6ba1021e8 Cooker NeverCookDirectories: Extend the nevercook of ExternalActors folder to include that relative path in all plugins in addition to /Game.
#rb Patrick.Enfedaque
#rnx
#preflight 62855cdb882a1e5cd7921b90

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 20268787 via CL 20268840 via CL 20268860
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20270507 by matt peters in ue5-main branch]
2022-05-18 19:01:15 -04:00
Matt Peters
cbcd67543d Cooker: Fix incorrect too-short waiting period being used for the cooker-blocked warning messages.
#rb None, trivial
#rnx
#preflight None, trivial
#fyi CarlMagnus.Nordin

[CL 20269632 by Matt Peters in ue5-main branch]
2022-05-18 18:09:12 -04:00
Matt Peters
6f3a384395 Cooker: Remove branch for CookOnTheFly from DecideNextCookAction. Implement the reduction of latency it was providing by changing the value of CookRetryBusyTime when in CookOnTheFly.
Fix precision issues with some times used in polling that were stored as floats instead of doubles.
#rb CarlMagnus.Nordin
#rnx
#preflight 6285674e614041edb7a85a12

[CL 20269102 by Matt Peters in ue5-main branch]
2022-05-18 17:50:42 -04:00
Matt Peters
b0c4684a38 CookOnTheFly: Move SaveAssetRegistry to before BeginCookPackageWriters.
#rb None, buildbreak fix
#rnx
#preflight 62855b98614041edb7a55419

[CL 20268522 by Matt Peters in ue5-main branch]
2022-05-18 17:11:34 -04:00
Matt Peters
02ba0b5b70 ZenStoreWriter: Fix assertion "Trying to add package that is already pending" if the cooker commits a package a second time while the first commit of the package is queued for asynchronous completion aand has not yet reached CommitPackageInternal.
This is necessary because as of CL 20151503, the cooker can commit the same package multiple times if the first save failed due to a timeout.

#fyi CarlMagnus.Nordin
#rnx
#rb Per.Larsson
#preflight 62853ba90f0d5076dee74ced

[CL 20265738 by Matt Peters in ue5-main branch]
2022-05-18 14:49:11 -04:00
Matt Peters
2da7663883 Cooker: Move shader file creation in CookByTheBook and CookOnTheFly to occurr immediately after BeginCookSandbox, since later steps in each mode can depend on the files being present.
Merge duplicated code in SaveGlobalShaderMapFiles and InitializeShadersForCookOnTheFly.
#preflight 62840a0d185a31bd39e4261d

[CL 20250859 by Matt Peters in ue5-main branch]
2022-05-17 17:06:00 -04:00
PJ Kack
97d39a7820 ZenCOTF: Fix for missing global shader files in zen file manifest (fallout from 20015198)
#jira none
#rb carlmagnus.nordin
#rnx
#preflight 62820da67d727306f20ab9fa

[CL 20221279 by PJ Kack in ue5-main branch]
2022-05-16 04:55:35 -04:00
Matt Peters
7032a26f23 SavePackage: Add IsCachedCookedPlatformDataLoaded calls; it is required by the contract of BeginCacheForCookedPlatformData. Add a Timeout result to return to the cookerto wait if any IsCachedCookedPlatformDataLoaded calls are incomplete.
#rb Francis.Hurteau
#rnx
#preflight 627c2ef47535c51f4c93b510

[CL 20151503 by Matt Peters in ue5-main branch]
2022-05-11 18:03:33 -04:00
charles bloom
5c59fd06ae clarify use of GetAllTextureFormats vs GetTextureFormats
GetAllTextureFormats is only used by COTF for version checking
could deprecate then remove it entirely

#rb none
#preflight 62797695e02e157f3e6a021b

[CL 20110473 by charles bloom in ue5-main branch]
2022-05-09 16:26:01 -04:00
josh adams
3d25a4ef0c - Un-reverting the FConfigContext and other cleanup change, as this fixes the plugin ini issue that was caused by missing changes from another changelist
#rb paul.chipchase
#p4v-preflight-copy 20033354
#preflight 6271b0c9e56311b5d8517c55

[CL 20074731 by josh adams in ue5-main branch]
2022-05-06 09:40:03 -04:00
josh adams
0a1910e517 [Backout] - CL20029165
#fyi Josh.Adams
Original CL Desc
-----------------------------------------------------------------
- Adding FConfigContext which is used to repalce LoadExternalIniFile, LoadLocalIniFile, etc, as well as have localized data for all configs read on a thread (like the other platform configs loaded in the editor)
- The Load*IniFile functions will create a Context, but eventually those APIs will go away and the Context will be the only way to load ini files
- Simplified some of the ini loading code, like removing the HierarchyCache (it wasn't helping editor load times, and added much complexity, and was not thread-safe, and it shouldn't actually be helpful because all the calls to Load*IniFile should eventually be replaced with either GConfig or FCOnfigCacheIni::ForPlatform(), which won't need to re-read in files
- Ini reading time actually went down due to the simplification, including Cache removal
- Left in old code for now behing a #define (USE_CONTEXT) in case something goes wrong
- Added in VERIFY_CONTEXT mode which I used to run original and Context modes and compare (including preflighting builds) (I also added a Compare function that we may want to keep around to use for future debugging)
- Added a separate set of config layers for plugins which speeds up plugin parsing, but also will fix the issue with BaseEngine.ini vs DefaultEngine.ini in Engine vs Project plugins (this shows how Contexts can bring useful information down into the guts - however a later change will enable it)
- Once this is all seen to be working, I will clean up the non-Context functions, and some globals vs static members, etc
#rb paul.chipchase
#jira none
#preflight 62716c1c5e6ce673f452005a

[CL 20033253 by josh adams in ue5-main branch]
2022-05-03 17:45:29 -04:00
Matt Peters
be11c56f83 Cooker: Fix assertion when cooking cooked editor DLC; SessionPlatforms are not yet available from the PlatformManager so we need to pass in the BeginCookContext to get them.
#rb Zousar.Shaker
#rnx
#preflight 627187c345186d6d29f49f0d

[CL 20031330 by Matt Peters in ue5-main branch]
2022-05-03 15:57:49 -04:00
Josh Adams
fd47cd56d9 - Adding FConfigContext which is used to repalce LoadExternalIniFile, LoadLocalIniFile, etc, as well as have localized data for all configs read on a thread (like the other platform configs loaded in the editor)
- The Load*IniFile functions will create a Context, but eventually those APIs will go away and the Context will be the only way to load ini files
- Simplified some of the ini loading code, like removing the HierarchyCache (it wasn't helping editor load times, and added much complexity, and was not thread-safe, and it shouldn't actually be helpful because all the calls to Load*IniFile should eventually be replaced with either GConfig or FCOnfigCacheIni::ForPlatform(), which won't need to re-read in files
- Ini reading time actually went down due to the simplification, including Cache removal
- Left in old code for now behing a #define (USE_CONTEXT) in case something goes wrong
- Added in VERIFY_CONTEXT mode which I used to run original and Context modes and compare (including preflighting builds) (I also added a Compare function that we may want to keep around to use for future debugging)
- Added a separate set of config layers for plugins which speeds up plugin parsing, but also will fix the issue with BaseEngine.ini vs DefaultEngine.ini in Engine vs Project plugins (this shows how Contexts can bring useful information down into the guts - however a later change will enable it)
- Once this is all seen to be working, I will clean up the non-Context functions, and some globals vs static members, etc
#rb paul.chipchase
#jira none
#preflight 62716c1c5e6ce673f452005a

[CL 20029165 by Josh Adams in ue5-main branch]
2022-05-03 14:13:03 -04:00
Matt Peters
9f84646586 CookOnTheFlyServer - StartCookByTheBook refactoring part 3.
1) Split StartCookByTheBook into subfunctions.
2) Rename some functions to match the new naming convention I've made for StartCookByTheBook helpers.
3) Make StartCookByTheBook and StartCookOnTheFly/AddCookOnTheFlyPlatform share subfunctions rather than duplicate code. Make Cancel and CookByTheBookFinished share subfunctions.
4) Remove up front allocation of MapDependencyGraphs; we can construct them on demand in CookByTheBookFinished.
#rb Zousar.Shaker
#rnx
#preflight 62703c3b220f89f0ad528cf6

[CL 20015198 by Matt Peters in ue5-main branch]
2022-05-02 16:31:37 -04:00
Matt Peters
3165e4a460 CookOnTheFlyServer - StartCookByTheBook refactoring part 2.
1) Remove GenerateAssetRegistry, it is no longer needed.
2) Factor shadercode out of StartCookByTheBook into a subfunction and move the portion that occurs after cleaning the sandbox to the end of StartCookByTheBook to give its asynchronous operations more time to complete.
3) Move CancelCookByTheBook out of the scoped cooktimer so it can clear the cook timers.
4) Move CookOnTheFly status logs into UpdateDisplay.
#rb Zousar.Shaker
#rnx
#preflight 626af9f8b17dd9121b5797e4

[CL 19971385 by Matt Peters in ue5-main branch]
2022-04-28 18:46:05 -04:00
Matt Peters
8c444d6487 CookOnTheFlyServer - StartCookByTheBook refactoring part 1.
1) Add CookSettings metadata to the cook directory so we can invalidate iterative cooks when a cook parameter has changed to a value incompatible with the previous cook.
2) Split BeginCookSandbox into two parts so we can calculate whether the build is iterative earlier and use it in functions that need to run before deleting the previous cook results.

#rb Zousar.Shaker
#rnx
#preflight 626ac681b17dd9121b425c6c
#preflight 626af9f8b17dd9121b5797e4

[CL 19968382 by Matt Peters in ue5-main branch]
2022-04-28 16:41:13 -04:00