Change consist of only forward declaration and additional includes
#preflight 63789c1de30d438849c48188
#rb none
[CL 23218412 by henrik karlsson in ue5-main branch]
* Fixed so all headers that are supposed to be included independently can compile (there are headers that must be included through other headers)
* Added HEADER_UNIT_SKIP to headers that must be included through others
* Added HEADER_UNIT_UNSUPPORTED to headers that are not supported by platform but is part of the include list anyway (files in Clang folder and pthread files).
* Added HEADER_UNIT_UNSUPPORTED to headers that have bad dependencies and include files from modules they don't depend on. (KeyChainUtilities.h and PackageWriter.h)
#preflight 637281309e3bea8079feb733
#rb devin.doucette
[CL 23127795 by henrik karlsson in ue5-main branch]
Takeaways. Try to use "friend" on functions/operators that are frequently overloaded EXCEPT if they are in a templated type that is frequently instantiated. So do not put friends in TMap, TSet, TObjectPtr etc, this will slow down compile times. There is a break-even somewhere and hard to tell where it is but taking a class that is templatized on character type probably don't matter either way and then it is nicer to use a friend since that simplies error messages when compiler can't resolve functions/operators.
If it is possible to use member functions instead of friend that is the best option in terms of compile time performance. With c++20 you only have to write operator==(Foo, Bar) and the compiler will automatically provide operator==(Bar, Foo), operator!=(Foo, Bar) and operator!=(Bar, Foo).
Changes in this changelist involes
* Making operator<< friends in non-template types and not friends in template types
* Making operator==/!= members where possible and if not possible moved out if type is a frequently instantiated templated type.
#preflight 636970f5376a9cd6a80da54a
#rb steve.robb
[CL 23038965 by henrik karlsson in ue5-main branch]
Moved GetTypeHash function to be hidden friend instead of put directly in global namespace.
Note that the function/operator needs to be fully inlined in the type or placed in the cpp. If the function is added as friend but then implemented outside the type then hidden friend optimization won't work.
This should improve compile time somewhat according to msvc devs.
#rb Steve.Robb
#preflight 6360b7052b5338aceb26471b
[CL 22889837 by henrik karlsson in ue5-main branch]
- This change removes the explicit usage of the package resource manager when loading bulk data from loose files or the Editor Domain. A new
I/O dispatcher backend reads data from the package resource manager with a new I/O chunk type constructed from a package name/package segment. When
serializing bulk data the I/O chunk type is set to the correct chunk type depending on whether the package is cooked or not.
#rb CarlMagnus.Nordin, Paul.Chipchase
#jira none
#preflight 633bdbc66b10157eac8e1807
[CL 22523877 by Per Larsson in ue5-main branch]
- PCD3D_SM5 and PCD3D_SM6 had a unique situation of having identical (bytecode-wise and output hash-wise) shaders (raytracing is allowed in PCD3D_SM5). This resulted in those IoStore chunks being deduplicated, however, the actual contents were not identical due to optional data (which isn't included in the shader hash, see UE-166389).
- This fix makes sure we don't use just the shader hash alone to produce IoStore chunk hashes. Hash will now include the shader size and the shader format name.
- A few early guards added here and there to detect the problem early.
Also fixed a memory leak in UnrealPak pointed by CM.
#rb [at]Yuriy.ODonnell, [at]CarlMagnus.Nordin, [at]Christopher.Waters, [at]Jason.Nadro
#jira UE-165785
#preflight 633f3beed718720d2d58f37e
[CL 22402802 by arciel rekman in ue5-main branch]
grouped inherited LLM tag, mem tag and trace metadata into FInheritedContext to simplify their usage, avoid forgetting using all of them together, and to hide the ugly conditional compilation. Added FInheritedContext to higher level tasks instead of the low-level tasks to allow batch task launch to capture the inherited context once per batch
#rb ionut.matasaru
#preflight 633c47792f4bf32d31687192
[CL 22331272 by Andriy Tylychko in ue5-main branch]
#rb CarlMagnus.Nordin, Paul.Chipchase
#jira none
#preflight 62fdf44787319bacfb44bdcb
- deprecated and replaced the usage of StartAsyncLoading and CreateStreamingRequestForRange with a new batch API for requesting bulk data. The
new API fits better with the I/O dispatcher API and does not assume that the bulk data resides in the same I/O chunk. The new API also
removes state handling away from the bulk data instances.
[CL 21439218 by Per Larsson in ue5-main branch]
Headers are updated to contain any missing #includes needed to compile and #includes are sorted. Nothing is removed.
#ushell-cherrypick of 21065896 by bryan.sefcik
#preflight 62d4b1a5a6141b6adfb0c892
#jira
#ROBOMERGE-OWNER: Bryan.sefcik
#ROBOMERGE-AUTHOR: bryan.sefcik
#ROBOMERGE-SOURCE: CL 21150156 via CL 21151754 via CL 21154719
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)
#ROBOMERGE-CONFLICT from-shelf
[CL 21181076 by Bryan sefcik in ue5-main branch]
Core public headers still will contain CoreMinimal.h if any included it before my IWYU changes.
#preflight 62c73594756222ced4e48707
[CL 20991367 by bryan sefcik in ue5-main branch]
Things to note regarding this pass:
* No includes are currently being removed from public headers.
* Any private file that has a #if is currently not being updated.
#preflight 62c703bbd13fac04f11da948
[CL 20985655 by bryan sefcik in ue5-main branch]
Things to note regarding this pass:
* No includes are currently being removed from public headers.
* Any private file that has a #if is currently not being updated.
#rb Devin Doucette and Marc Audy
#preflight 62c62086756222ced497ff95
#preflight 62c626d12f2d04691814d331
#preflight 62c630107e606620fdd64e31
#preflight 62c635c6756222ced49a13cf
[CL 20979027 by bryan sefcik in ue5-main branch]
Can now provide reference iostore containers to reuse compressed chunks from. If a match is found on *the decompressed data*, instead of recompressing the blocks, they are read off of disk. This allows tweaks of the compressor algorithm without introducing changes as the runtime still sees the exact same data. Additionally this allows for fairly dramatic staging speedups as nvme speeds are significantly faster than high effort compressions. This is distinct from the DDC compression because:
1) DDC compression ties in the compressor version/method
2) We are explicitly interested in chunks that are deployed to end users, not merely cached for speed.
To facilitate this, several changes were made to IoStore:
FIoStoreReader now directly reads from IFileHandles* instead of routing through the GenericPlatformFile async read system, as that system is sensitive to build #defines and can result in constant file opens under load (indeed, for anything not a .pak file, every read is an open/close).
Cold file cache read speed improvements from ~140MB/s to ~1 GB/s. Hot is more.
Additionally:
FIoStoreReader switched to UE::Tasks from taskgraph for tasks in order to facilitate task retraction during waits as the previous ReadAsync call was trivial to deadlock when called from worker threads due to its use of TFuture<>.
FIoStoreReader::ReadCompressed now returns the compressed blocks as they were on disk - padded to AES encryption block size.
#rb fabian.giesen
#rb jeff.roberts
#preflight 627586dcf77c9c2b543d4d8b
[CL 20086673 by Dan Thompson in ue5-main branch]
-- Shaders must be sorted prior to inlining, as they are inserted in to a sorted list.
-- Break ties sorting shaders on disk with chunkid (hash)
-- Sort the shader map hashes in the package store entry
-- Pick the lowest LoadOrderFactor instead of the first for shaders that are used in more than one package.
-- Fix -WriteFinalOrder to actually write the final order as well as all chunks.
#rb fabian.giesen, charles.bloom
#preflight 627441a7de15651eeb416fbf
[CL 20065025 by Dan Thompson in ue5-main branch]
LoadDisplayNames() 1.06s -> 0.24s when async loading the name batch in CachedAssetRegistry.bin during editor startup of an internal project
* Introduce FDisplayNameEntryId to avoid cache misses in FName::GetComparisonIdFromDisplayId() lookups
* Force name batch serialization to use display entry ids to help enforce cooking determinism
* Fix determinism problem when loading then saving via FPackageStoreNameMapBuilder
* Inline FNameEntryId::FromUnstableInt()
* Remove slow IsNumbered() check in CreateFromDisplayId when not using 4B FName
* Remove expensive heap sort by sorting up front instead
#rb rob.millar,matt.peters,pj.kack
#preflight 62664bd06968f18d5137d419
[CL 19899104 by Johan Torp in ue5-main branch]
