Commit Graph

72 Commits

Author SHA1 Message Date
robert manuszewski
d1443992e1 Deprecating ANY_PACKAGE.
This change consists of multiple changes:

Core:
- Deprecation of ANY_PACKAGE macro. Added ANY_PACKAGE_DEPRECATED macro which can still be used for backwards compatibility purposes (only used in CoreUObject)
- Deprecation of StaticFindObjectFast* functions that take bAnyPackage parameter
- Added UStruct::GetStructPathName function that returns FTopLevelAssetPath representing the path name (package + object FName, super quick compared to UObject::GetPathName) + wrapper UClass::GetClassPathName to make it look better when used with UClasses
- Added (Static)FindFirstObject* functions that find a first object given its Name (no Outer). These functions are used in places I consider valid to do global UObject (UClass) lookups like parsing command line parameters / checking for unique object names
- Added static UClass::TryFindType function which serves a similar purpose as FindFirstObject however it's going to throw a warning (with a callstack / maybe ensure in the future?) if short class name is provided. This function is used  in places that used to use short class names but now should have been converted to use path names to catch any potential regressions and or edge cases I missed.
- Added static UClass::TryConvertShortNameToPathName utility function
- Added static UClass::TryFixShortClassNameExportPath utility function
- Object text export paths will now also include class path (Texture2D'/Game/Textures/Grass.Grass' -> /Script/Engine.Texture2D'/Game/Textures/Grass.Grass')
- All places that manually generated object export paths for objects will now use FObjectPropertyBase::GetExportPath
- Added a new startup test that checks for short type names in UClass/FProperty MetaData values

AssetRegistry:
- Deprecated any member variables (FAssetData / FARFilter) or functions that use FNames to represent class names and replaced them with FTopLevelAssetPath
- Added new member variables and new function overloads that use FTopLevelAssetPath to represent class names
- This also applies to a few other modules' APIs to match AssetRegistry changes

Everything else:
- Updated code that used ANY_PACKAGE (depending on the use case) to use FindObject(nullptr, PathToObject), UClass::TryFindType (used when path name is expected, warns if it's a short name) or FindFirstObject (usually for finding types based on user input but there's been a few legitimate use cases not related to user input)
- Updated code that used AssetRegistry API to use FTopLevelAssetPaths and USomeClass::StaticClass()->GetClassPathName() instead of GetFName()
- Updated meta data and hardcoded FindObject(ANY_PACKAGE, "EEnumNameOrClassName") calls to use path names

#jira UE-99463
#rb many.people
[FYI] Marcus.Wassmer
#preflight 629248ec2256738f75de9b32

#codereviewnumbers 20320742, 20320791, 20320799, 20320756, 20320809, 20320830, 20320840, 20320846, 20320851, 20320863, 20320780, 20320765, 20320876, 20320786

#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 20430220 via CL 20433854 via CL 20435474 via CL 20435484
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v949-20362246)

[CL 20448496 by robert manuszewski in ue5-main branch]
2022-06-01 03:46:59 -04:00
paul chipchase
f85fad88ff It is now possible to disable virtualization for all payloads owned by a specific asset type by adding the name of the asset to [Core.ContentVirtualization]DisabledAsset string array in the engine ini file.
#rb Per.Larsson
#rnx
#jira UE-151377
#preflight 628364050039ea57a52d6989

### Virtualization
- [Core.ContentVirtualization] in the engine ini file now supports an array called 'DisabledAsset' which can be used to name asset types that should not virtualize their payloads.
-- By default (in BaseEngine.ini) we have disabled the StaticMesh asset as we know it will crash if a payload is missing and the SoundWave asset as it still is pending testing.
- This new way to disable virtualization is data driven. The older hard coded method has not been removed but will likely be reworked in a future submit.
- Now when an editor bulkdata is adding it's payload to the package trailer builder during package save it will poll the virtualization system with a call to the new method ::IsDisabledForObject by passing in it's owner.
-- If the owner is valid and was present in the 'DisabledAsset' array then the method will return true and the EPayloadFlags::DisableVirtualization flag will be applied.

### Package Trailer
- The pre-existing functionality of enum EPayloadFilter has been moved to a new enum EPayloadStorageType as will only filter payloads based on their storage type.
- EPayloadFilter has been modified to filter payloads based on functionality although at the moment the only thing it can filter for is to return payloads that can be virtualized, it is left for future expansion.
- EPayloadFlags has been reduced to a uint16 with the remaining 2bytes being turned into a new member EPayloadFilterReason.
- This new member allows us to record the exact reason why a payload is excluded from virtualization. If it is zero then the payload can virtualize, otherwise it will contain one or more reasons as to why it is being excluded. For this reason the enum is a bitfield.
- Added overloads of ::GetPayloads and ::GetNumPayloads that take EPayloadFilter rather than a EPayloadStorageType
- Added wrappers around all AccessMode types for FLookupTableEntry.
- FPackageTrailerBuilder has been extended to take a EPayloadFilterReason so that the caller can already provide a reason why the payload cannot be virtualized.
-- As a future peace of work this will probably be changed and we will ask the caller to pass in the owner UObject pointer instead and then we will process the filtering when building the package trailer to a) keep all of the filtering code in one place b) keep the filtering consistent

### PackageSubmissionChecks
- The virtualization process in  will now request the payloads that can be virtualized from the package trailer, which respects the new payload flag, rather than requesting all locally stored payloads.

### UObjects
- There is no need for the SoundWave or MeshDescription classes to opt out of virtualization on construction. This will be done when the package is saved and is now data driven rather than being hardcoded.

### DumpPackagePayloadInfo
- The command has been updated to also display the filter reasons applied to each payload

[CL 20240971 by paul chipchase in ue5-main branch]
2022-05-17 07:54:28 -04:00
Matt Peters
46290e86ee AssetDataGatherer: Change serialization format of dependencies in the editor's AssetDataGatherer cache to improve load performance.
#rb Zousar.Shaker
#rnx
#preflight 6245a64e637925b5d3c6c729

[CL 19571741 by Matt Peters in ue5-main branch]
2022-03-31 09:16:55 -04:00
Matt Peters
6bc3029111 PackageReader and EditorDomain: add all objects of type UScriptStruct to the list of ImportedClasses. UScriptStruct objects might be used for serialization even if they are not the class of an export in the package. One example of this is UDataTable's RowStruct.
#rb Zousar.Shaker
#rnx
#jira UE-146751
#preflight 623a2cb789625f0612b655ae

[CL 19471743 by Matt Peters in ue5-main branch]
2022-03-22 16:31:12 -04:00
Matt Peters
0e341a856d Add DoesPackageExistOnDisk to the AssetRegistry. Use it to check whether the AssetRegistry thinks a package exists without having to copy the entire FAssetPackageData.
In a future change we will add tracking of extension and of package name capitalization to FAssetPackageData, and will use DoesPackageExistOnDisk to replace the more expensive PackageResourceManager::TryMatchCaseOnDisk calls that occur when loading a package.

Change FPackageDatas to use DoesPackageExist, which is based on the AssetPackageDatas, rather than using GetAssetsByPackageName, because GetAssetsByPackageName can have false negatives from packages that are missing assets and can have false positives from in-memory packages that have added their Assets to the AssetRegistryState via ProcessLoadedAssetsToUpdateCache.

#jira UE-144429
#jira UE-143860
#rb Johan.Torp
#rnx
#preflight 622f75d26131e07703ac806b

[CL 19374576 by Matt Peters in ue5-main branch]
2022-03-14 13:20:30 -04:00
steve robb
deebf593a2 Fix for mixing SUBOBJECT_DELIMITER_CHAR with non-TCHARs - when used in a ternary in particular, it caused the result to become promoted to an integer.
#rb martin.ridgers
#preflight 6225fe2d825c6abb99fa5d1c

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19284148 via CL 19295463 via CL 19304889 via CL 19305094
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19347125 by steve robb in ue5-main branch]
2022-03-10 21:02:28 -05:00
matt peters
06cd5d762d FAssetDataGatherer/FPackageReader: Fix assets created by FPackageReader to match the LongPackageName the AssetDataGatherer has for them. This is important in cases of mounting ambiguity, such as when running UE5 without a project, and /Game and /Engine mount to the same localpath ../../../Engine/Content.
#jira UE-143620
#preflight 621e4025f1206ae3ea5b7807
#rnx
#rb Leon.Huang
#lockdown Mark.Lintott


#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 19207514 via CL 19209514 via CL 19211747 via CL 19211756 via CL 19212108
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)

[CL 19223983 by matt peters in ue5-main branch]
2022-03-02 13:04:48 -05:00
paul chipchase
c7cad5cf29 Improve user visibility for packages with virtualized data by adding "Has Virtualized Data" to the tool tip in the content browser as well as adding an asset filter to hide all packages without virtualized data.
#rb Johan.Torp, Lauren.Barnes
#jira UE-135423
#rnx
#preflight 620374f674604bc6b1a62882

- The filter can be found in Filter->Other Filters -> Virtualized Data
- Since we only want to store if a package has virtualized data or not we can just use a bit in FAssetPackageData::Flags

[CL 18951020 by paul chipchase in ue5-main branch]
2022-02-11 08:04:32 -05:00
matt peters
d8a113d3ab #jira UE-135316
AssetRegistry: Fix crash in package reader if a content directory is unmounted in between queing the package and reading it.
#rb Johan.Torp
#rnx

#ROBOMERGE-AUTHOR: matt.peters
#ROBOMERGE-SOURCE: CL 18256769 in //UE5/Release-5.0/... via CL 18256777
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18256786 by matt peters in ue5-release-engine-test branch]
2021-11-22 08:01:32 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00
paul chipchase
ecdec9bf35 Add an UE5 specific version EUnrealEngineObjectUE5Version to be used for global changes instead of EUnrealEngineObjectUEVersion. By splitting and storing both version numbers we allow for hypothetical future UE4 changes that will not conflict when merged to UE5.
#rb CarlMagnus.Nordin
#rnx
#tests Ran overnight preflights on several platforms, opened/cooked/staged/ran the oldest version of InfiltratorDemo that can be downloaded (4.11)

### ObjectVersion
- Add a new version enum EUnrealEngineObjectUE5Version.
-- This version number starts at 1000 which leaves more than enough for for EUnrealEngineObjectUEVersion to be expanded
- Even though very few changes (if any at all) to EUnrealEngineObjectUE4Version are expected there is a static assert to make sure that EUnrealEngineObjectUEVersion::AUTOMATIC_VERSION never overtakes EUnrealEngineObjectUE5Version::INITIAL_VERSION.
- Add a struct FPackageFileVersion that wraps around the version numbers and is used to store them instead of raw int32 values which was done before. This should make it easier to add new version numbers in the future if we desire (although this will cause problems in places that serialize the struct directly)

### FPackageFileSummary
- Adding a new entry to CurrentLegacyFileVersion at value -8 which shows the UE5 version being added. This lets us make the changes without needing to submit anything to UE4 Main.
- When loading a package that does not have a UE5 version, it will remain at 0.
- Added ::IsFileVersionTooOld and ::IsFileVersionTooNew to replace hardcoded tests in the code base for version validity. This will make it easier to make changes in the future.
- A few months ago most of the accessors of the version number were deprecated in favour of a version that did not contain the Engine number (ie UE4Ver -> UEVer in Archive) but to work with these changes the renamed methods now will return or accept the version as FPackageFileVersion rather than int32.  The old UE4 methods will remain deprecated and direct licensees to use the new methods.

### Archive
- Now stores the version as a FPackageFileVersion rather than int32

### LinkerLoad
- Reports the larger version number if we detect a higher version number than we support. Note that this could cause an issue if the UE4 version is ever raised but helps keep the code simple.

### AssetData
- Need to add a new version here to manage existing data that only has the UE4 version

### EditorDomain
- We do not need to version the format, we can just invalidate existing editor domain entries via EditorDomainVersion

### EditorServer
- When reporting that a package is too old we report the UE4 version as that is the only version that can be older than VER_UE4_OLDEST_LOADABLE_PACKAGE
- When reporting that a package is too new it can be either the UE4 or the UE5 version so we print them together "UE4Ver|UE5Ver"

### ContentCommandlets
- The min and max resave versions have been kept as a single value, you will not be able to resave against different UE4 and UE5 versions at the same time. It doesn't seem like a useful feature and would greatly increase the complexity of the code.
- We will also only report the file version as a single value.

### ManifestUObject
- This class was setting an older obsolete version on purpose to try and maintain compatibility with older clients so we need to provide a way to create an older UE4 only version that will leave the UE5 version as unset.

### NetworkPlatformFile
- I was unable to test the code path in FNetworkPlatformFile::ProcessServerCachedFilesResponse as I am unsure how to run the game in a mode that will actually use it.
- When reading an older "CookedVersion.txt" that was saved with a single version, the reads will fail and this will count as a version change in the code so that all of the existing files will be deleted. The existing code would not give the user a log message when this happens and given the very small time window where this might happen caused by this change I have opted to leave this alone and not add any additional logging.
- If we do detect a version mismatch we will still only log the version number as a single version.

### CookOnTheFlyServer
- We now add each version number to the IniVersionMap rather than merge the version and license version as a key/value pair. This allows us to a) use both the UE4 and UE5 version numbers b) we now log a warning that the version values don't match when it is changed, previously since it was a key value we would log a warning about an additional setting instead.
-- I also added "vs" to the log message when values are mismatched to make the space between the two values being printed clearer.

#ROBOMERGE-OWNER: paul.chipchase
#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 17549459 via CL 17550236 via CL 17550238 via CL 17550582
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v870-17433530)
#ROBOMERGE[STARSHIP]: UE5-Main

[CL 17550583 by paul chipchase in ue5-release-engine-test branch]
2021-09-17 07:04:55 -04:00
matt peters
52eccb16ee PackageReader: Fix typo: accidental remnant of previous version of the for loop.
#rb None, trivial
#rnx

#ROBOMERGE-SOURCE: CL 16333186 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v804-16311228)

[CL 16333215 by matt peters in ue5-release-engine-test branch]
2021-05-14 15:04:48 -04:00
matt peters
dbc465cb75 EditorDomain: Add list of ImportedClasses for each package to the AssetRegistry's FAssetPackageData.
#rb Johan.Torp
#rnx

#ROBOMERGE-SOURCE: CL 16328609 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v804-16311228)

[CL 16328632 by matt peters in ue5-release-engine-test branch]
2021-05-14 08:33:53 -04:00
mark lintott
2fa0ec20f0 Removal of UE4 references from code base
#rb trivial

[CL 16005742 by mark lintott in ue5-main branch]
2021-04-14 11:13:44 -04:00
mark lintott
bd61859350 #jira 112822
#rb johan.torp
Removal of UE4 references in Archive.h

[CL 16002350 by mark lintott in ue5-main branch]
2021-04-14 05:14:13 -04:00
Marc Audy
01b7c9f4f5 Merge UE5/RES @ 15958325 to UE5/Main
This represents UE4/Main @ 15913390 and Dev-PerfTest @ 15913304

[CL 15958515 by Marc Audy in ue5-main branch]
2021-04-08 14:32:07 -04:00
mark lintott
24420693b6 #jira UE-111195
Updated references to UE4 to Unreal in NetworkFileConnection.cpp
This required references to GPackageFileUE4Version and GPackageFileLicenseeUE4Version to also be updated to GPackageFileUnrealVersion and GPackageFileLicenseeUnrealVersion accordingly

#ROBOMERGE-SOURCE: CL 15850452 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v786-15839533)

[CL 15851651 by mark lintott in ue5-main branch]
2021-03-29 11:38:01 -04:00
Matt Peters
b18493397a AssetRegistry - Add CustomVersion information to AssetRegistry's PackageData information. Add IAssetRegistry::WaitOnPackage.
These features will be used in the future by the Editor Domain.
#rb Johan.Torp
#rnx

[CL 15391145 by Matt Peters in ue5-main branch]
2021-02-11 16:00:51 -04:00
Francis Hurteau
cfc1fe9f21 Narrow the GIsSavingPackage check that prevent object creation and FindObject calls to be restricted to the actual package being saved by using a temporary package flag
#rb Matt.Peters, Robert.Manuszewski

#ROBOMERGE-OWNER: Francis.Hurteau
#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 15378239 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v771-15082668)
#ROBOMERGE-CONFLICT from-shelf

[CL 15378989 by Francis Hurteau in ue5-main branch]
2021-02-10 15:37:45 -04:00
julien lheureux
6636097aa4 Fixed outstanding issue in the PackageReader that would assume ownership of archive even if unwarranted;
Exposed a way to read the FAssetData from a file in the AssetDataGatherer api
Augmented IAssetRegistry api to support getting registry data from a filename, not only an archive
Replaced use of "old" code so that there is one path for asset data recovery only.

#rb sebastien.lussier, jeanfrancois.dube, jamie.dale

[CL 15164467 by julien lheureux in ue5-main branch]
2021-01-22 12:31:57 -04:00
Marc Audy
bc88b73a29 Merge Release-Engine-Staging to Main @ CL# 15151250
Represents UE4/Main @ 15133763

[CL 15158774 by Marc Audy in ue5-main branch]
2021-01-21 16:22:06 -04:00
Marc Audy
50a3d7d368 Merge Release-Engine-Staging to Main @ CL# 14467590
This represents UE4/Main @ 14432125 + some cherrypick fixes

[CL 14468207 by Marc Audy in ue5-main branch]
2020-10-09 22:42:26 -04:00
Marcus Wassmer
3b81cf8201 Merging using //UE5/Main_to_//UE5/Release-Engine-Staging @14384769
autoresolved files
#rb none

[CL 14384911 by Marcus Wassmer in ue5-main branch]
2020-09-24 00:43:27 -04:00
Marc Audy
11f5b21210 Merging //UE5/Release-Engine-Staging @ 13752110 to Main (//UE5/Main)
#rnx

[CL 13753156 by Marc Audy in ue5-main branch]
2020-06-23 18:40:00 -04:00
ryan durand
0f0464a30e Updating copyright for Engine Runtime.
#rnx
#rb none


#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869210 via CL 10869511 via CL 10869900
#ROBOMERGE-BOT: (v613-10869866)

[CL 10870549 by ryan durand in Main branch]
2019-12-26 14:45:42 -05:00