Live++ reads object files at startup for game modules, and assigns unique ids to each compiland (used to disambiguate static variables). When compiling the patch, these compilands are modified to reference a unique id for the unity blob, causing the variables to be reconstructed.
Solution is to generate a JSON file to each output directory containing object files containing the mapping, and to use that to assign compiland ids at startup.
#rb none
#jira UE-74036
#ROBOMERGE-SOURCE: CL 6455253 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 6455273 by ben marsh in Main branch]
#rb none
#jira UE-73075
#ROBOMERGE-SOURCE: CL 6017285 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 6017289 by ben marsh in Main branch]
#jira
#rb none
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: ben.marsh
#ROBOMERGE-SOURCE: CL 5993252 via CL 5993257 via CL 5995286
[CL 5995562 by ben marsh in Main branch]
#rb none
#jira UE-72524
#ROBOMERGE-SOURCE: CL 5988868 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5988908 by ben marsh in Main branch]
Encoding improvements
* New hash table implementation
--- Move away from 16-bit hashes since we need more than 64k buckets to hold 2M entries efficiently
--- Change to CityHash64, which is faster and stronger
--- Remove hardcoded max limit
* NAME_INDEX changed from contigouos int to monotonically increasing int
--- Opens up for future deduplication schemes that are better than number suffix dedup.
--- Saves some memory since we don't need to maintain a contiguous array
--- Typed up to cause compile errors when used as integer directly
* Avoid touching data repeatedly, normal path only does single hash of data
* New constructor that allows supplying string length up front
* Avoid dynamic allocations in string conversions >= 128 characters
* Avoid extra copying when splitting numbers
* More efficient IsPureAnsi check
* Only do one global lazy initialization call instead of multiple
* Switch to faster RW locks
* Switch from single lock to sharded hash map with separate locks
* Memory optimizations that reduces per entry overhead: 4 bytes slots, 2 byte headers and don't store null terminator
Improved API & documentation
* Document that IsValid() rarely makes sense
* Hide global state such as GetNames()
* Reduce amount of implementation details visible in header
* NameTypes.h size down by ~1/3 while adding documentation, stronger type safety and new APIs
Future possibilities
* Memory savings: Removing public NAME_INDEX and global FName array allows using the 32-bit FNameEntryId for arbitrary deduplication schemes. This can save both actual stored strings memory by deduplication and half the size of FName instances from 8B to 4B in shipping / test configs by removing the number part.
* Implementation can be tweaked further, for instance could persist 32-bit slot index hash inside slot to increase encoding performance in development / debug at the cost of memory.
Perf & mem results for internal project:
* Editor startup wall time: 12% speedup, 22.0s -> 19.5s
--- Warm disk and asset registry cache
--- Some gains from nametable serialization improvements
--- Some of these gains might be from orthogonal asset discovery optimizations
* Win64 Test Client memory usage with ~400k names: 30.3Mb -> 22.2Mb
#rb steve.robb, pj.kack
#jira UE-59973
[CL 5774657 by Johan Torp in Dev-Core branch]
Now stores the linker environment block in the live coding manifest, and adds it to the cache before executing the linker.
#rb none
#jira UE-72324
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: ben.marsh
#ROBOMERGE-SOURCE: CL 5740602 in //UE4/Release-4.22/... via CL 5740765
#ROBOMERGE-BOT: CORE (Main -> Dev-Core)
[CL 5743844 by ben marsh in Dev-Core branch]
Now stores the linker environment block in the live coding manifest, and adds it to the cache before executing the linker.
#rb none
#jira UE-72324
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: ben.marsh
#ROBOMERGE-SOURCE: CL 5740602 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5740765 by ben marsh in Main branch]
- This feature allows different chunk layout on different platforms.
- Most mapping work is done in AssetRegistryGenerator.
- Terminology of ChunkID and Pakchunk Index are now separated and they are no longer the same if chunk id to pakchunk index mapping is defined. The ChunkIDs in FAssetData is not renamed to keep consistency, but it actually is pakchunk index. FAssetData.ChunkIDs is used for ChunkInstall.
- Chunks with encryption key guid or unique assetregistry name are not allowed to be mapped. There are code to filter out those chunks and log errors while initializing mapping
- CheckChunkAssetsAreNotInChild() and ResolveChunkDependencyGraph() is skipped when making build for XboxOne and PS4, since they are not needed. If for some reason, we need chunk dependency on console builds, we need to implement per-platform dependency graph, since now we support different chunk layout on different platforms.
- Disabled a few pak file overide rules for console builds, since they should only be applied to builds with Kairos chunk layout.
#test A preflight was kicked and tested on PS4 and XboxOne. No new issue was found.
#rb Daniel.Lamb
#ROBOMERGE-SOURCE: CL 5470414 via CL 5474050
[CL 5474308 by hongyi yu in Main branch]
#rb none
#jira
#ROBOMERGE-SOURCE: CL 5437980 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5437990 by ben marsh in Main branch]
* Removed code to invalidate makefiles when adding new source files. UBT should be reliable enough to make this determination itself nowadays, and ignored -invalidatemakefilesonly argument was causing modules to be recompiled.
* Fixed incorrect config section name when determining whether to allow hot reload from IDE. Now prevents hot reload from IDE when live coding is enabled.
* Added error message when trying to add a new class with Live Coding enabled.
* Added error messages when trying to start Live Coding after a hot reload has taken place.
* Added error messages when trying to hot reload with Live Coding enabled.
#jira UE-71253
#rb none
#ROBOMERGE-SOURCE: CL 5403464 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5403485 by ben marsh in Main branch]
#rb none
#jira UE-71399
#ROBOMERGE-SOURCE: CL 5365389 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5368509 by ben marsh in Main branch]
#rb none
#jira UE-71379
#ROBOMERGE-SOURCE: CL 5362145 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5368489 by ben marsh in Main branch]
#rb none
#jira UE-71269
#ROBOMERGE-SOURCE: CL 5361379 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5368488 by ben marsh in Main branch]
#rb none
#jira UE-71270
#ROBOMERGE-SOURCE: CL 5361279 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5368485 by ben marsh in Main branch]
#rb none
#jira UE-71359
#ROBOMERGE-SOURCE: CL 5361201 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5368484 by ben marsh in Main branch]
