#rb david.harvey
#ROBOMERGE-SOURCE: CL 6581404 via CL 6581406 via CL 6591268
#ROBOMERGE-BOT: (v351-6581450)
[CL 6591502 by ben woodhouse in Main branch]
Params:
-csvabtest="<cvar>=<value0>,<value1>..."
[-csvabtestStatFrameOffset=<N frames>]
[-csvabtestswitchduration=<N frames>]
[-csvabtestfastcvarset] - skips cvar syncs
#rb anthony.bills
#ROBOMERGE-SOURCE: CL 6054267 via CL 6054966 via CL 6055780
[CL 6055837 by ben woodhouse in Main branch]
[REVIEW]
#rb Josh.Adams
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: chris.bunner
#ROBOMERGE-SOURCE: CL 5833466 via CL 5833467 via CL 5833468 via CL 5837320 via CL 5838091
#ROBOMERGE-BOT: CORE (Main -> Dev-Core)
[CL 5879225 by chris bunner in Dev-Core branch]
Encoding improvements
* New hash table implementation
--- Move away from 16-bit hashes since we need more than 64k buckets to hold 2M entries efficiently
--- Change to CityHash64, which is faster and stronger
--- Remove hardcoded max limit
* NAME_INDEX changed from contigouos int to monotonically increasing int
--- Opens up for future deduplication schemes that are better than number suffix dedup.
--- Saves some memory since we don't need to maintain a contiguous array
--- Typed up to cause compile errors when used as integer directly
* Avoid touching data repeatedly, normal path only does single hash of data
* New constructor that allows supplying string length up front
* Avoid dynamic allocations in string conversions >= 128 characters
* Avoid extra copying when splitting numbers
* More efficient IsPureAnsi check
* Only do one global lazy initialization call instead of multiple
* Switch to faster RW locks
* Switch from single lock to sharded hash map with separate locks
* Memory optimizations that reduces per entry overhead: 4 bytes slots, 2 byte headers and don't store null terminator
Improved API & documentation
* Document that IsValid() rarely makes sense
* Hide global state such as GetNames()
* Reduce amount of implementation details visible in header
* NameTypes.h size down by ~1/3 while adding documentation, stronger type safety and new APIs
Future possibilities
* Memory savings: Removing public NAME_INDEX and global FName array allows using the 32-bit FNameEntryId for arbitrary deduplication schemes. This can save both actual stored strings memory by deduplication and half the size of FName instances from 8B to 4B in shipping / test configs by removing the number part.
* Implementation can be tweaked further, for instance could persist 32-bit slot index hash inside slot to increase encoding performance in development / debug at the cost of memory.
Perf & mem results for internal project:
* Editor startup wall time: 12% speedup, 22.0s -> 19.5s
--- Warm disk and asset registry cache
--- Some gains from nametable serialization improvements
--- Some of these gains might be from orthogonal asset discovery optimizations
* Win64 Test Client memory usage with ~400k names: 30.3Mb -> 22.2Mb
#rb steve.robb, pj.kack
#jira UE-59973
[CL 5774657 by Johan Torp in Dev-Core branch]
[FYI] ben.woodhouse
#ROBOMERGE-SOURCE: CL 5713957 via CL 5713961 via CL 5714341 via CL 5714931 via CL 5716577
[CL 5716660 by bart hawthorne in Main branch]
[FYI] ben.woodhouse
#ROBOMERGE-SOURCE: CL 5696133 via CL 5696188 via CL 5696201 via CL 5696240 via CL 5697550
[CL 5697679 by bart hawthorne in Main branch]
#rb none
#JIRA
#ROBOMERGE-OWNER: Juan.Canada
#ROBOMERGE-AUTHOR: juan.canada
#ROBOMERGE-SOURCE: CL 5317515 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5335728 by Juan Canada in Main branch]
