Commit Graph

365 Commits

Author SHA1 Message Date
andriy tylychko
d938d346bd fixed crash in AsyncFileHandle with unimplemented EnsureCompletion()
#jira UE-195894
#rb danny.couture, francis.hurteau

[CL 28313586 by andriy tylychko in ue5-main branch]
2023-09-28 07:09:26 -04:00
nick edwards
220ccc4b3e FIoDirectoryIndexReader::IterateDirectoryIndex: Use FStringViews and TStringBuilders instead of FStrings to avoid potentially unnecessary heap allocations
FPakPlatformFile::ForeachPackageInIostoreWhile: Update usage of IterateDirectoryIndex, and use overload of TryConvertFilenameToLongPackageName using string views and builders to avoid allocations

#rb elias.lozadabenavent
#rb anton.dunchev

[CL 28282945 by nick edwards in ue5-main branch]
2023-09-27 14:10:02 -04:00
justin marcus
895bb59e4d GFP state machine can now detect when its possible to force monolithic shader library loading which is faster if we know we don't need to search for chunked shader libraries.
- Can be disabled with CVarAllowForceMonolithicShaderLibrary.
- FGameFeaturePreMountingContext can disable shader library loading in cases where we may want to control it externally.
- Install bundle manager now tracks if a bundle contains chunks
- GetPakchunkIndexFromPakFile now uses string views and string builders instead of temporary strings
- GetPakchunkIndexFromPakFile is no longer duplicated in IPlatformFilePak
- FPakPlatformFile::HandleMountPakDelegate no longer does a linear search to find the just mounted pak file


[REVIEW] [at]Dave.Belanger [at]Thomas.Ross [at]Bob.Tellez

[CL 28049947 by justin marcus in ue5-main branch]
2023-09-20 15:11:58 -04:00
matt peters
2d8cad97a6 PakFiles and FileManager: Add IPlatformFile::FDirectoryVisitor::ShouldVisitLeafPath for directory visitors that are ignoring some files, so that FPakFile::ValidateDirectorySearch knows that invalid-to-request files were not requested despite being present in the searched directories.
#rnx
#rb Zousar.Shaker

[CL 28012483 by matt peters in ue5-main branch]
2023-09-19 16:51:05 -04:00
matt peters
cca6524868 PlatformFilePak: Move IPlatformFile directory API functions into cpp file for readability and compile time improvements. Remove functions deprectated on or before 4.27.
#rb None, trivial
#rnx

[CL 27971653 by matt peters in ue5-main branch]
2023-09-18 15:54:58 -04:00
matt peters
0538198ab4 PakFiles: Downgrade the Warning: MissingPrunedPakFile from error to warning until we implement a fix for the spurious instances of the error.
#rnx
#rb None, trivial

[CL 27935012 by matt peters in ue5-main branch]
2023-09-15 17:31:46 -04:00
Matt Peters
e69f086889 CookOnTheFly2: Remove duplicate warnings for missing packages requested multiple times, and remove warnings for editor-only packages that are found due to unsolicited loads on the cooker but were not requested by the client. Add packagename to warnings about missing packages.
#jira UE-184879
#rnx
#rb Dan.Englebrecht

[CL 27593637 by Matt Peters in ue5-main branch]
2023-09-05 08:41:53 -04:00
david harvey
35c367d801 Centralize ENABLE_PLATFORM_CHUNK_INSTALL declarations into GenericPlatformChunkInstall.h
#rnx
#rb trivial

[CL 27480709 by david harvey in ue5-main branch]
2023-08-30 06:36:54 -04:00
matt peters
678e02de1a PakFiles: Add logging of the callstack for errors about directory searches that return incomplete results because of pakfilepruning.
#rnx
#rb Dan.Thompson

[CL 27067146 by matt peters in ue5-main branch]
2023-08-14 08:42:51 -04:00
robert millar
c5a7662189 Speculative fixes for hang detected in ReleaseOldReaders
Acquire list of pak files and then release pak-list lock.
Only release if we can acquire the reader lock without waiting.

#rb none

[CL 26864080 by robert millar in ue5-main branch]
2023-08-04 18:41:29 -04:00
david harvey
847fd70841 Add a new experimental bundle source that uses the platform chunk installer, via some optional new functions in the chunk installer interface.
#jira UE-146398
#rnx
#rb Justin.Marcus

[CL 26733411 by david harvey in ue5-main branch]
2023-08-01 04:46:51 -04:00
spoiltos
420553ff43 PR #10540: Fixed assert message in IPlatformFilePak.cpp
#rb zahra.nikbakht

[CL 26183285 by spoiltos in ue5-main branch]
2023-06-22 11:10:00 -04:00
henrik karlsson
5db685f97d [Engine]
* Moved dllexport from type to methods/staticvar in all Engine runtime code. This improves compile times, memory and performance in dll builds

[CL 26082269 by henrik karlsson in ue5-main branch]
2023-06-17 18:13:06 -04:00
eric knapik
bc3b5dd786 #jira: none
Adding some Pak initialization and deinitialization logging to track down a horde only ensure.

#rb: trivial

[CL 25795014 by eric knapik in ue5-main branch]
2023-06-05 12:36:27 -04:00
Wojciech Krywult
d1baf24722 AsyncIO: Fixed incorrect handling of the STAT_AsyncFileMemory stat.
I've reviewed all code using async requests and we had multiple places incorrectly handling the stat: some placed failed to decrease the counter, while others would decrease it when they shouldn't. The main cause was the fact that when we create an async request and pass nullptr as the target memory, the request allocates the memory itself and increases the STAT_AsyncFileMemory. However, when we call GetReadResults() on such requests, the responsibility is on the caller to balance the calls and decrease the stat, which is not very obvious and leads to the mentioned bugs. On top of that, when we call GetReadResults() we may not have the knowledge as to whether the returned memory was allocated by the request or not (which may affect whether we should touch the stat at all).

Resolved by adding IAsyncReadRequest::ReleaseMemoryOwnershipImpl member, which may be used by request implementations to do the proper clean up (balance the stat). This way requests take the full responsibility for their changes and users of the requests don't need to know their implementation details.

#preflight none
#rb Patrick.Laflamme
#jira UE-185064

[CL 25664733 by Wojciech Krywult in ue5-main branch]
2023-05-29 12:11:37 -04:00
dan thompson
5ecef89101 Validating index loading for upacks.
#jira UE-173038
#rb charles.bloom
#preflight 646d0bc96c2a2532b1747645

[CL 25593251 by dan thompson in ue5-main branch]
2023-05-23 18:33:52 -04:00
dan thompson
aba11d90b1 Sanitize inputs for UncompressCopyFile
#jira UE-173032
#jira UE-173029
#jira UE-173028
#rb fabian.giesen
#preflight 646bccd2576becd7b57ad6e1

[CL 25572218 by dan thompson in ue5-main branch]
2023-05-22 19:17:22 -04:00
per larsson
f72cc84a5a I/O - move slab allocator to internal for reuse between backends
#rb PJ.Kack
#preflight 645dc3106b4bdec267c9cfd2

[CL 25444366 by per larsson in ue5-main branch]
2023-05-12 03:50:13 -04:00
pj kack
12064cb1d3 IoDispatcher: Improve shutdown logic.
First reset the GIoDispatcher pointer.
Then stop the IoDispatcher thread and wait for all pending requests to finish.
Then shutdown the backends.
Then destroy the IoDispatcher itself.

#rb per.larsson
#rnx
#preflight 644bbbe91c2846595c308b2a

[CL 25298458 by pj kack in ue5-main branch]
2023-05-02 03:28:46 -04:00
matt peters
b6bd711eaa PakFiles: If FChunkCacheWorker's construction fails and the FChunkCacheWorker is invalid, fail the entire Pak by returning nullptr from CreatePakReader rather than allowing creation of an FSignedArchiveReader that will crash in FChunkCacheWorker::CheckSignature.
#rb Robert.Millar
#rnx
#preflight 6446b821dc71c13446eed5d1

[CL 25175481 by matt peters in ue5-main branch]
2023-04-24 23:21:31 -04:00
bryan sefcik
c7696eaa09 Updated several modules to set UnsafeTypeCastWarningLevel to WarningLevel.Error and not WarningLevel.Warning. The use of WarningLevel.Warning makes separate shared PCHs.
#preflight 643718baf12d5de7058cd78c

[CL 25015908 by bryan sefcik in ue5-main branch]
2023-04-12 17:09:36 -04:00
carlmagnus nordin
ecfc01c258 ZenLoader: Include preload dependencies in package headers instead of relying on predetermined export bundle dependencies. This means that each package can be treated in isolation when staging. Also removes the need to keep track of export and bundle counts in the package store which saves some memory.
#preflight 642e8838441e8b2f5506dbb4
#rb pk.kack,per.larsson,francis.hurteau

[CL 24944233 by carlmagnus nordin in ue5-main branch]
2023-04-06 10:50:30 -04:00
carlmagnus nordin
462cc2a438 IoStore: Include imported package names in the header data. This removes the need to keep track of uncooked packages in the package store for editor builds.
#rnx
#rb pj.kack
#preflight 64256a01b01a25d6a044758d

[CL 24869323 by carlmagnus nordin in ue5-main branch]
2023-03-31 07:20:51 -04:00
Nicholas Frechette
471ad92304 Prevent memory mapping compressed or encrypted memory with IoStore
#jira UE-146471
#rb carlmagnus.nordin
#preflight 641da9bdc44ce895fc9ac839

[CL 24778890 by Nicholas Frechette in ue5-main branch]
2023-03-24 10:06:39 -04:00
Andriy Tylychko
33551a6eaf deprecated FCoreDelegates::OnFileOpenedForReadFromPakFile because it's thread-safe but is used concurrently. Added a thread-safe version FCoreDelegates::GetOnFileOpenedForReadFromPakFile() and switched all use cases to the new one
#rb francis.hurteau
#preflight 641d9f8d7a393e211af9829b
#preflight 641d9f8d7a393e211af9829b

[CL 24778456 by Andriy Tylychko in ue5-main branch]
2023-03-24 09:22:49 -04:00