Optimized Collection code invoked at each source control update to avoid comparing large set to detect if the in-memory collection is different from the collection on disk.
#rb Julien.StJean
#preflight 6346e2cf663dafe4162e8c9e
[CL 22505265 by patrick laflamme in ue5-main branch]
Removed redundant private include paths from build.cs files.
Fixed include paths to be relative to the private or public folders.
Hid or removed includes that reached into other private module folders.
Updated PublicInclude paths when necessary.
#jira
#preflight 631e281694758d0bf2ea1399
[CL 21960082 by bryan sefcik in ue5-main branch]
SizeType will be deprecated in a subsequent change.
#rb Zousar.Shaker
#rnx
#preflight 61b8dde95d1b7767a13a5a70
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18456907 in //UE5/Release-5.0/... via CL 18456915
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v898-18417669)
[CL 18456926 by devin doucette in ue5-release-engine-test branch]
#ROBOMERGE-AUTHOR: bob.tellez
#ROBOMERGE-SOURCE: CL 18327503 via CL 18327514 via CL 18327529 via CL 18328464 via CL 18328574
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)
[CL 18328663 by bob tellez in ue5-release-engine-test branch]
Supplied by David.Bollo of The Coalition.
Fixes some non unity compile errors
[FYI] David.Bollo
#jira none
#ROBOMERGE-SOURCE: CL 17206575 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)
[CL 17206608 by dan phillips in ue5-release-engine-test branch]
- Parallelize collection loading in FCollectionManager::LoadCollections
- Parallelize FName registration to cut on hashing cost in Collection::Load
- Leverage FStringView where possible in Collection::Load
- 987ms -> 238ms for collection loading during editor startup
- 987ms -> 696ms with -onethread
#rnx
#tests generated md5 of all loaded collections content before and after the change
#rb Bob.Tellez
#ROBOMERGE-SOURCE: CL 13054205 via CL 13054206 via CL 13054211
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v686-13045012)
[CL 13054213 by danny couture in Main branch]
#rnx
#rb none
#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)
[CL 10870584 by ryan durand in Main branch]
Some displayed error messages about paths being too long are more informatives and include full path and current/maximum length allowed.
#rb none
[CL 10119068 by Gines Hidalgo in Dev-Editor branch]
Revamped collection UI:
- Uses a more modern design.
- Adds the asset count to the collection view.
- Adds the collections more prominently in asset tooltips.
Revamped sources panel UI:
- Added a unified search for all aspects of the sources panel (paths, favorites, and collections).
- Added a switcher for the paths and collections views, which auto-switches as you browse to paths or collections.
- Collections are now visible by default, rather than hidden behind a View Option.
Improved Content Browser auto-complete:
- Now lists collection and meta-data tag names, as well as the old class names.
- Tokenizes the search terms to allow auto-complete on partial statements.
- Note: Involves some breaking API changes to SAssetSearchBox to allow more complex filtering and auto-complete rules.
Added scripting API support:
- Collections can now be created, destroyed, and manipulated via Python or Editor Utilities (editor-only, see the AssetTagsSubsystem API).
- Collections can be queried both in-editor and at runtime (if cooked).
Added support for optionally "cooking" collections for use at runtime:
- This only works for static collections, and adds their information to the asset registry meta-data.
- See the AssetRegistry settings in BaseEngine.ini (bTagAllCollections, CollectionsToIncludeAsTags, and CollectionsToExcludeAsTags).
- No collections are cooked by default.
#rb Chris.Gagnon
[CL 8847950 by Jamie Dale in Dev-Editor branch]
We've had several cases where the previously default alphabetical sorting caused performance issues when it wasn't needed. We now require you to explicitly choose either slow alphabetical sorting or fast non-alphabetical order.
The fast order does not decode, access or compare actual string data, but the order is not stable between process runs.
#rb steve.robb
[CL 5915265 by Johan Torp in Dev-Core branch]
Encoding improvements
* New hash table implementation
--- Move away from 16-bit hashes since we need more than 64k buckets to hold 2M entries efficiently
--- Change to CityHash64, which is faster and stronger
--- Remove hardcoded max limit
* NAME_INDEX changed from contigouos int to monotonically increasing int
--- Opens up for future deduplication schemes that are better than number suffix dedup.
--- Saves some memory since we don't need to maintain a contiguous array
--- Typed up to cause compile errors when used as integer directly
* Avoid touching data repeatedly, normal path only does single hash of data
* New constructor that allows supplying string length up front
* Avoid dynamic allocations in string conversions >= 128 characters
* Avoid extra copying when splitting numbers
* More efficient IsPureAnsi check
* Only do one global lazy initialization call instead of multiple
* Switch to faster RW locks
* Switch from single lock to sharded hash map with separate locks
* Memory optimizations that reduces per entry overhead: 4 bytes slots, 2 byte headers and don't store null terminator
Improved API & documentation
* Document that IsValid() rarely makes sense
* Hide global state such as GetNames()
* Reduce amount of implementation details visible in header
* NameTypes.h size down by ~1/3 while adding documentation, stronger type safety and new APIs
Future possibilities
* Memory savings: Removing public NAME_INDEX and global FName array allows using the 32-bit FNameEntryId for arbitrary deduplication schemes. This can save both actual stored strings memory by deduplication and half the size of FName instances from 8B to 4B in shipping / test configs by removing the number part.
* Implementation can be tweaked further, for instance could persist 32-bit slot index hash inside slot to increase encoding performance in development / debug at the cost of memory.
Perf & mem results for internal project:
* Editor startup wall time: 12% speedup, 22.0s -> 19.5s
--- Warm disk and asset registry cache
--- Some gains from nametable serialization improvements
--- Some of these gains might be from orthogonal asset discovery optimizations
* Win64 Test Client memory usage with ~400k names: 30.3Mb -> 22.2Mb
#rb steve.robb, pj.kack
#jira UE-59973
[CL 5774657 by Johan Torp in Dev-Core branch]
