Commit Graph

104 Commits

Author SHA1 Message Date
robert manuszewski
d1443992e1 Deprecating ANY_PACKAGE.
This change consists of multiple changes:

Core:
- Deprecation of ANY_PACKAGE macro. Added ANY_PACKAGE_DEPRECATED macro which can still be used for backwards compatibility purposes (only used in CoreUObject)
- Deprecation of StaticFindObjectFast* functions that take bAnyPackage parameter
- Added UStruct::GetStructPathName function that returns FTopLevelAssetPath representing the path name (package + object FName, super quick compared to UObject::GetPathName) + wrapper UClass::GetClassPathName to make it look better when used with UClasses
- Added (Static)FindFirstObject* functions that find a first object given its Name (no Outer). These functions are used in places I consider valid to do global UObject (UClass) lookups like parsing command line parameters / checking for unique object names
- Added static UClass::TryFindType function which serves a similar purpose as FindFirstObject however it's going to throw a warning (with a callstack / maybe ensure in the future?) if short class name is provided. This function is used  in places that used to use short class names but now should have been converted to use path names to catch any potential regressions and or edge cases I missed.
- Added static UClass::TryConvertShortNameToPathName utility function
- Added static UClass::TryFixShortClassNameExportPath utility function
- Object text export paths will now also include class path (Texture2D'/Game/Textures/Grass.Grass' -> /Script/Engine.Texture2D'/Game/Textures/Grass.Grass')
- All places that manually generated object export paths for objects will now use FObjectPropertyBase::GetExportPath
- Added a new startup test that checks for short type names in UClass/FProperty MetaData values

AssetRegistry:
- Deprecated any member variables (FAssetData / FARFilter) or functions that use FNames to represent class names and replaced them with FTopLevelAssetPath
- Added new member variables and new function overloads that use FTopLevelAssetPath to represent class names
- This also applies to a few other modules' APIs to match AssetRegistry changes

Everything else:
- Updated code that used ANY_PACKAGE (depending on the use case) to use FindObject(nullptr, PathToObject), UClass::TryFindType (used when path name is expected, warns if it's a short name) or FindFirstObject (usually for finding types based on user input but there's been a few legitimate use cases not related to user input)
- Updated code that used AssetRegistry API to use FTopLevelAssetPaths and USomeClass::StaticClass()->GetClassPathName() instead of GetFName()
- Updated meta data and hardcoded FindObject(ANY_PACKAGE, "EEnumNameOrClassName") calls to use path names

#jira UE-99463
#rb many.people
[FYI] Marcus.Wassmer
#preflight 629248ec2256738f75de9b32

#codereviewnumbers 20320742, 20320791, 20320799, 20320756, 20320809, 20320830, 20320840, 20320846, 20320851, 20320863, 20320780, 20320765, 20320876, 20320786

#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 20430220 via CL 20433854 via CL 20435474 via CL 20435484
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v949-20362246)

[CL 20448496 by robert manuszewski in ue5-main branch]
2022-06-01 03:46:59 -04:00
Matt Peters
81a3ae5b81 Cooker: Multiprocess support: Remove some deprecated code and move some out of place functions that should not be executed on CookWorker into the proper calling function that will be called on the director only.
#rb Zousar.Shaker
#rnx
#preflight 629135e11b686da53f41b463

[CL 20398652 by Matt Peters in ue5-main branch]
2022-05-27 16:48:59 -04:00
Matt Peters
8e80f6dc13 Cooker PakChunk files: Suppress the creation of empty pakchunkN.txt files for empty chunks. It causes a performance problem when the project is using high chunkid values for organization while leaving lower values unused.
#rb Justin.Marcus
#rnx
#preflight 6290f5e1b83292836e08957c

[CL 20394815 by Matt Peters in ue5-main branch]
2022-05-27 12:31:23 -04:00
Dan Thompson
cd73707429 Update package writers to gather package hashes internally, then have CookByTheBookFinished copy them to asset package datas.
Add iostore chunk hashes to the asset registry during cook.
Remove old hashing code, which was extensive. This moves the async writes in to UE::Task from the old AsyncWorkSequence().
#rb Matt.Peters
#rb Jeff.Roberts
#preflight 624b24a5f73c316f68303946

[CL 19611812 by Dan Thompson in ue5-main branch]
2022-04-04 13:26:27 -04:00
marc audy
ce6733551a #UE Fixing warning about ambiguous class that is emitted when a world partition actor package has GetClass() called on it during the asset registry generation part of the cook. GetClass() is using ANY_PACKAGE to search for the class, which is wrong when called on assets whose type may be generated from a blueprint, since blueprints can have the same name in different paths
#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: bob.tellez
#ROBOMERGE-SOURCE: CL 19439879 via CL 19444852 via CL 19445671 via CL 19445727
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)

[CL 19445893 by marc audy in ue5-main branch]
2022-03-19 18:34:22 -04:00
matt peters
716be0f4a2 FAssetData: Callsites need to handle AssetData.GetClass() returning null. Added IsInstanceOf(UClass*) for the common operation GetClass() && GetClass->IsChildOf(BaseClass).
#jira UE-146521
#rb Zousar.Shaker
#rnx
#preflight 6234e1b4ed772061b62b6c66
#lockdown Simon.Tourangeau

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 19440852 in //UE5/Release-5.0/... via CL 19441616
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)

[CL 19442531 by matt peters in ue5-main branch]
2022-03-18 18:56:39 -04:00
Wojciech Krywult
11352afe9e Packaging: Fixed incorrect generation of layer information for subchunks leading to package generator crashing with an exception.
Subchunks are generated when having MaxChunkSize parameter set to a non-zero value in the project packaging properties. The problem was caused by the generated pakchunklayers.txt having information only for the main chunks, but not for subchunks. The package generator assumes that pakchunklayers.txt has the same number of lines as the generated pakchunklist.txt (storing a list of all chunks and subchunks).

This issue is a regression introduced in CL 7637286, which modified the code to output layer information for empty chunks.

#jira UE-146062
#rb dave.barrett
#preflight 6234adc40820efd0946b55eb

[CL 19437131 by Wojciech Krywult in ue5-main branch]
2022-03-18 12:31:03 -04:00
Dan Thompson
d412b366a3 Adding support to writing platform specific tags to the asset registry during cook by the book.
FArchive now holds a "CookData" blob that contains relevant information for cooking. The TargetPlatform has been moved to this structure, and it has a reference to a CookContext that can hold data generated during the cook that doesn't get serialized to the uasset (in this case, the asset registry tags).
#rb Matt.Peters
#preflight 62151bc4797dbbeb472af77d
#preflight 621011f6ff52bfecfc032b15
#preflight 62152777141b500e17eb24ef

[CL 19076066 by Dan Thompson in ue5-main branch]
2022-02-22 13:32:02 -05:00
justin marcus
414cc52a5e Fix NoSaveDevAR option also keeping AllChunksInfo.csv from being written.
#rb Josh.Adams
#preflight

#ROBOMERGE-AUTHOR: justin.marcus
#ROBOMERGE-SOURCE: CL 19000716 via CL 19003250 via CL 19003309 via CL 19003327 via CL 19008122 via CL 19008726
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)

[CL 19008802 by justin marcus in ue5-main branch]
2022-02-15 20:37:08 -05:00
Matt Peters
11cd6f4774 #jira UE-140189
Cooker: Fix tagging of assets in collections; AssetData tags now always require a non-empty value.
#preflight 62040121bf7362cd77f3c167
#rb Jamie.Dale
#rnx

[CL 18921179 by Matt Peters in ue5-main branch]
2022-02-09 13:13:19 -05:00
josh adams
9d69636403 - Preload the DevAR based on a #define (ASSETREGISTRY_ENABLE_PREMADE_REGISTRY_IN_EDITOR) as well as REquiresCookedData check
- Allow for skipping of saving the DevelopmentAssetREgistry.bin (useful when cooking DLC)
- Don't initialize the TemporaryState in AssetRegistry when cooking DLC
- Allow for some delaying of blocking on AsyncAR
#preflight 61fb43075a7645dc85e82ffa
#rb matt.peters,daniel.lamb

#ROBOMERGE-OWNER: josh.adams
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 18843186 via CL 18844583 via CL 18844592 via CL 18844604 via CL 18847702 via CL 18848026
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v910-18824042)

[CL 18848072 by josh adams in ue5-main branch]
2022-02-03 15:00:32 -05:00
francis hurteau
d262daac13 Added support to Loose File Package Writer to write multiple package output per input
Added similar support to the package store manifest
Zen Store support will be added separately
Added some extra data to linker object resources to be used by the iostore package optimizer to properly support optional package chunks.

Multiple cooking outputs is used to support "optional objects" which are editor data package exports that would be put in separate iostore containers than the normal cooked game content.

#jira UE-129801
#rb Matt.Peters, CarlMagnus.Nordin
#preflight 61e5d3bb873f2ea48f48166c

#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 18639742 in //UE5/Release-5.0/... via CL 18639751 via CL 18639758
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v900-18638592)

[CL 18639762 by francis hurteau in ue5-main branch]
2022-01-18 09:56:23 -05:00
aurel cordonnier
34f55d3a4a Merge from Release-Engine-Test @ 17946149 to UE5/Main
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17949667 by aurel cordonnier in ue5-main branch]
2021-10-27 15:14:40 -04:00
Matt Peters
9948113093 AssetRegistryGenerator - update function names and comments for clarity.
Split ExtraFlavor features in two to make them clearer.
Move FixupPackageDependenciesForChunks from SaveManifests into FinalizeChunkIDs since it needs to happen even when not saving manifests.
Set DiskSize to -1 for failed package saves in UpdateAssetRegistryPackageData, since that is part of updating assetdata for the package.

[CL 17628885 by Matt Peters in ue5-main branch]
2021-09-27 07:18:14 -04:00
Matt Peters
fe9a7566db AssetRegistryGenerator - update syntax to current patterns used on the core team. No change in behavior.
#rb Zousar.Shaker
#rnx

[CL 17556606 by Matt Peters in ue5-main branch]
2021-09-17 15:11:07 -04:00
Matt Peters
83e3dd1f3b CookedPackageWriter iterative builds - change to use FAssetRegistryState as the list of previous files. Load the AssetRegistry from the AssetRegistry.bin file even when using ZenStore for the package storage.
#rb Per.Larsson
#rnx

[CL 17551332 by Matt Peters in ue5-main branch]
2021-09-17 09:32:26 -04:00
Matt Peters
75c591648c DevelopmentAssetRegistry.bin: Reduce duplication by making a function to return this string.
#rb Per.Larsson
#rnx

[CL 17506665 by Matt Peters in ue5-main branch]
2021-09-14 13:30:20 -04:00
aurel cordonnier
e3f7878676 Merge from Release-Engine-Test @ 17462327 to UE5/Main
This represents UE4/Main @17430120 and Dev-PerfTest @17437669

[CL 17463546 by aurel cordonnier in ue5-main branch]
2021-09-08 16:42:26 -04:00
Matt Peters
02dfa9a6ec Cooker: Hybrid Iterative cooks
Add the concept of pre-exploration of dependencies - searching assetregistry or target domain dependencies of the set of requested packages up front, to find all packages that will be cooked and allow us to do asynchronous and batch operations on them.
Convert FRequestCluster into an FPackageData container, holding packages in a substate of EPackageState::Request. This allows us to create a new RequestClusters when we encounter a package that was not found in the initial discovery.
Add hybriditerative mode for iterative cooks; this uses targetdomain keys to check for packages that exist from the previous cook, rather tracing the dependency graph to find all packages that transitively reference packages with modified guids.
Modify AssetRegistryGenerator to use a TMap of iteratively loaded packages that can be edied as packages cook, rather than using a const previous state; this supports hybrid iterative where we do not calculate the list of iterative packages until we encounter them in a request cluster.
#rb Zousar.Shaker
#rnx

[CL 17373295 by Matt Peters in ue5-main branch]
2021-08-31 14:20:16 -04:00
Matt Peters
69d6366855 CookOnTheFlyServer: Create FLooseCookedPackageWriter and refactor UCookOnTheFlyServer to use ICookedPackageWriter in all cases rather than having a separate path for writing cooked files to disk.
#rb CarlMagnus.Nordin
#rnx

[CL 17247861 by Matt Peters in ue5-main branch]
2021-08-20 10:56:53 -04:00
charles bloom
b5d07316b0 Log time in SaveAssetRegistry
#rb none

[CL 17136801 by charles bloom in ue5-main branch]
2021-08-11 11:49:26 -04:00
Per Larsson
6450577448 Iterative cooking support when using Zen loader/storage server
Zen: af7ff3f1c6

#rb CarlMagnus.Nordin, Stefan.Boberg
#jira none
#rnx

[CL 16755977 by Per Larsson in ue5-main branch]
2021-06-23 09:00:52 -04:00
aurel cordonnier
43fa62fcd8 Merge from Release-Engine-Test @ 16487383 to UE5/Main
This represents UE4/Main @ 16445039 and Dev-PerfTest @ 16444526

[CL 16488106 by aurel cordonnier in ue5-main branch]
2021-05-27 13:40:37 -04:00
aurel cordonnier
8eebe8841f Merge UE5/RET @ 16305968 to UE5/Main
This represents UE4/Main @ 16261013 and Dev-PerfTest @ 16259937

[CL 16306996 by aurel cordonnier in ue5-main branch]
2021-05-12 18:10:03 -04:00
aurel cordonnier
50944fd712 Merge UE5/RES @ 16162155 to UE5/Main
This represents UE4/Main @ 16130047 and Dev-PerfTest @ 16126156

[CL 16163576 by aurel cordonnier in ue5-main branch]
2021-04-29 19:32:06 -04:00