Commit Graph

301 Commits

Author SHA1 Message Date
CarlMagnus Nordin
63b8cbc9dd AsyncLoading2: Implemented loading of optional package segments
#rnx
#preflight 62627548ac29fcf6a213fe33
#rb francis.hurteau

[CL 19900368 by CarlMagnus Nordin in ue5-main branch]
2022-04-25 07:37:07 -04:00
Johan Torp
ef8e0c4660 Optimize WITH_CASE_PRESERVING_NAME FName loading
LoadDisplayNames() 1.06s -> 0.24s when async loading the name batch in CachedAssetRegistry.bin during editor startup of an internal project

* Introduce FDisplayNameEntryId to avoid cache misses in FName::GetComparisonIdFromDisplayId() lookups
* Force name batch serialization to use display entry ids to help enforce cooking determinism
* Fix determinism problem when loading then saving via FPackageStoreNameMapBuilder
* Inline FNameEntryId::FromUnstableInt()
* Remove slow IsNumbered() check in CreateFromDisplayId when not using 4B FName
* Remove expensive heap sort by sorting up front instead

#rb rob.millar,matt.peters,pj.kack
#preflight 62664bd06968f18d5137d419

[CL 19899104 by Johan Torp in ue5-main branch]
2022-04-25 03:48:33 -04:00
Matt Peters
f23ab4e7bb FPaths and PakPlatformFile: suppress the Pak warning mounting to unregistered paths when the mount path is GameFeatureRootPrefix.
#jira UE-149431
#rb Brandon.Schaefer
#rnx
#preflight 625efc29db15ac92dba1c193

[CL 19820566 by Matt Peters in ue5-main branch]
2022-04-19 18:41:54 -04:00
dave belanger
7cdeea2000 Support -UsePaks on monolithic editor with unccoked data
#rb Josh.Adams
#preflight 6238d7c910251d53d5636a1b

#ROBOMERGE-AUTHOR: dave.belanger
#ROBOMERGE-SOURCE: CL 19465146 via CL 19465653 via CL 19466852 via CL 19475582 via CL 19475844
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v935-19464907)

[CL 19476451 by dave belanger in ue5-main branch]
2022-03-23 01:26:26 -04:00
danny couture
158690e923 Reduce string allocations/copies in FindFilesAtPathInIndex
#rb Matt.Peters
#preflight 621f671231454c90ccf90fb5

[CL 19221346 by danny couture in ue5-main branch]
2022-03-02 09:33:34 -05:00
Josh Adams
720a0d8e74 - Fixed comment typo
#rb none
#preflight skip

[CL 19058889 by Josh Adams in ue5-main branch]
2022-02-18 18:15:29 -05:00
Josh Adams
f8f462876d - Added a -LookLooseFirst commandline option that tells pak file system to prefer loose files before looking in the pak, instead of falling back to loose when not found in the pak (this is useufl for quickly patching a pak with some local files for debugging)
#rb matt.peters
#jria none
#preflight 621016fa676ae0301347ed92

[CL 19057756 by Josh Adams in ue5-main branch]
2022-02-18 17:15:58 -05:00
sebastian thomeczek
638adf1a61 IPlatformFilePak:
Added additional check and error message that triggers when a pak file is mounted with a mount point that is not mounted to any root directory

#rb Matt.Peters
#jira none
#preflight 61f803fa114ec25fe0a22fa0

#ROBOMERGE-AUTHOR: sebastian.thomeczek
#ROBOMERGE-SOURCE: CL 18792180 in //UE5/Release-5.0/... via CL 18792252 via CL 18792985
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18793152 by sebastian thomeczek in ue5-main branch]
2022-01-31 13:25:06 -05:00
carlmagnus nordin
d91fbf9061 Removed option to run with package data in iostore and shaders in pak
Now that all asset data will be in either iostore or pak setting async io priority based on the iodispatcher request queue contents is no longer needed
#preflight 61f799286a7c7b134f2fe78c
#rnx
#rb pj.kack

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18787380 in //UE5/Release-5.0/... via CL 18787398 via CL 18787423
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18787426 by carlmagnus nordin in ue5-main branch]
2022-01-31 03:49:53 -05:00
dan thompson
17ae342aa6 Fix for spurious FileCache/CompressionBlockSize warning:
-- When not using iostore containers
-- When not using compression
#rb trivial
[FYI] carlmagnus.nordin
[FYI] charles.bloom
#preflight 61f43351801201ab3890cd22

#ROBOMERGE-AUTHOR: dan.thompson
#ROBOMERGE-SOURCE: CL 18774859 in //UE5/Release-5.0/... via CL 18774927 via CL 18775395
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18775464 by dan thompson in ue5-main branch]
2022-01-28 13:51:28 -05:00
justin marcus
7e6de992a0 Fix FFileIoStoreContainerFile::FilePath not being set in FFileIoStoreReader::Initialize. This made it impossible to unmount iostore containers which kept them from being deleted.
[REVIEW] [at]PJ.Kack [at]CarlMagnus.Nordin [at]Graeme.Thornton
#test Used VK Edit iteration
#preflight 61f1b574f8088a3d29944983

#ROBOMERGE-AUTHOR: justin.marcus
#ROBOMERGE-SOURCE: CL 18746124 via CL 18747191 via CL 18747196 via CL 18747200 via CL 18747760 via CL 18747947
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18747968 by justin marcus in ue5-main branch]
2022-01-26 19:09:35 -05:00
sebastian thomeczek
7ef35c3511 Adjust LogPakFile logging verbosity
- added VeryVerbose message when access to any file from ExcludedNonPakExtensions is prevented
- Introduced new Display message to print mounted pak file name and mount point, moved the corresponding logs that were already printing that information from the shadercodelibrary to verbose
- Moved most FPakReadRequest messages to VeryVerbose
- Moved PrimaryIndexSize PathHashIndexSize and FullDirectoryIndeSize to Verbose
- Moved OnPakFileMounted2Time to Verbose

#rb Matt.Peters, PJ.Kack
#jira none
#preflight 61f00f308c16f837224ada89

#ROBOMERGE-AUTHOR: sebastian.thomeczek
#ROBOMERGE-SOURCE: CL 18722494 in //UE5/Release-5.0/... via CL 18722496 via CL 18722552
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18722570 by sebastian thomeczek in ue5-main branch]
2022-01-25 10:08:12 -05:00
marc audy
0af59ec276 AsyncLoading2:
Add support for importing uncooked source packages from cooked packages
#rnx
#jira UE-136265, FORT-426538
#rb pj.kack
#preflight 61e6ff6a3778a195debba826

#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18655480 via CL 18664834 via CL 18665656 via CL 18665916 via CL 18667722 via CL 18667764 via CL 18668174
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v901-18665521)

[CL 18668352 by marc audy in ue5-main branch]
2022-01-19 18:28:42 -05:00
dave barrett
e3adea40cd revert 18571612 which causes initialization failure on multiple platforms
#preflight 61de37f2b97cb668809cd78c
#jira UE-138902
#rb none

#ROBOMERGE-AUTHOR: dave.barrett
#ROBOMERGE-SOURCE: CL 18581245 in //UE5/Release-5.0/... via CL 18581267 via CL 18581273
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v899-18417669)

[CL 18581274 by dave barrett in ue5-main branch]
2022-01-11 23:07:51 -05:00
sebastian thomeczek
b580fbb4ff Adjust LogPakFile and add error message on unreachable pak mount points
- Added additional check and error message that triggers when a pak file is mounted with a mount point that is not mounted to any root directory
- added VeryVerbose message when access to any file from ExcludedNonPakExtensions is prevented
- Introduced now Display message to print mounted pak file name and mount point, moved the corresponding logs that were already printing that information from the shadercodelibrary to verbose
- Moved most FPakReadRequest messages to VeryVerbose
- Moved PrimaryIndexSize PathHashIndexSize and FullDirectoryIndeSize to Verbose
- Moved OnPakFileMounted2Time to Verbose

#rb Matt.Peters, PJ.Kack
#jira none
#preflight 61dd8d95676009ad46512283

#ROBOMERGE-AUTHOR: sebastian.thomeczek
#ROBOMERGE-SOURCE: CL 18571612 in //UE5/Release-5.0/... via CL 18571622
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18571623 by sebastian thomeczek in ue5-release-engine-test branch]
2022-01-11 09:49:52 -05:00
dan thompson
f903f566d5 Add a warning for when the FileCache block size doesn't match any iostore container compressed block sizes.
#rb CarlMagnus.Nordin
#preflight 61dc76678f44ab439b39e987

#ROBOMERGE-AUTHOR: dan.thompson
#ROBOMERGE-SOURCE: CL 18562735 in //UE5/Release-5.0/... via CL 18562758
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18562781 by dan thompson in ue5-release-engine-test branch]
2022-01-10 13:43:28 -05:00
carlmagnus nordin
3f35cbfed3 IoDispatcher:
Restored container mounting error message log text
More detailed error messages when reading container headers

#rnx
#jira UE-137253
#rb pj.kack

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18475303 via CL 18475339 via CL 18475344 via CL 18475346 via CL 18475493 via CL 18481186
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18481210 by carlmagnus nordin in ue5-release-engine-test branch]
2021-12-16 19:09:32 -05:00
pj kack
11261e35d4 Add missing include to IoDispatcherFileBackendTypes.h
#rb none
#rnx
#preflight 61baeb7158796f05e155c12c

#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 18474142 in //UE5/Release-5.0/... via CL 18480613
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18480686 by pj kack in ue5-release-engine-test branch]
2021-12-16 18:20:49 -05:00
pj kack
00e60736c6 Add WindowsPlatformIoDispatcher module with a file backend platform implementation that uses DirectStorage for Windows
#jira none
#rb carlmagnus.nordin
#rnx
#preflight 61ba1db2590498f205a46dae
#preflight 61ba3212753a9d8c87bbb5dd

#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 18469239 in //UE5/Release-5.0/... via CL 18469248
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18469258 by pj kack in ue5-release-engine-test branch]
2021-12-15 13:59:37 -05:00
carlmagnus nordin
fdc3ad3cb7 Move IoDispatcherFileBackend from Core to PakFile
#rnx
#preflight 61b89ea22e65a1df0459b68a
#rb pj.kack

#ROBOMERGE-OWNER: carlmagnus.nordin
#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18465244 via CL 18465257 via CL 18465387 via CL 18465391 via CL 18465458 via CL 18465466
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)

[CL 18465469 by carlmagnus nordin in ue5-release-engine-test branch]
2021-12-15 03:24:48 -05:00
carlmagnus nordin
b783f6c882 SignedArchiveReader: Make sure to write some data to the output buffer even if the signatures are missing. Otherwise we could end up with a valid pak file magic number from a previous read and incorrectly consider the pak file valid.
#rnx
#rb pj.kack

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18378980 via CL 18379001 via CL 18379016 via CL 18379019 via CL 18379023 via CL 18379027 via CL 18434282 via CL 18435644
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18436609 by carlmagnus nordin in ue5-release-engine-test branch]
2021-12-10 19:18:37 -05:00
carlmagnus nordin
4a7d7d277c Implemented -checkpak support for iostore containers
#rnx
#rb pj.kack
#preflight 61a5dd60be6d0d4512d6cd04

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18322656 via CL 18322665 via CL 18323459 via CL 18323526 via CL 18323898 via CL 18323959
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)

[CL 18323988 by carlmagnus nordin in ue5-release-engine-test branch]
2021-11-30 09:00:43 -05:00
Marc Audy
0c3be2b6ad Merge Release-Engine-Staging to Test @ CL# 18240298
[CL 18241953 by Marc Audy in ue5-release-engine-test branch]
2021-11-18 14:37:34 -05:00
aurel cordonnier
fc542f6cfd Merge from Release-Engine-Staging @ 18081189 to Release-Engine-Test
This represents UE4/Main @18073326, Release-5.0 @18081140 and Dev-PerfTest @18045971

[CL 18081471 by aurel cordonnier in ue5-release-engine-test branch]
2021-11-07 23:43:01 -05:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00