#rb rune.stubbe
#jira none
#rnx
#ROBOMERGE-SOURCE: CL 13007135 in //UE4/Release-4.25/... via CL 13007150 via CL 13007159
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v682-12900288)
[CL 13007166 by jeff newquist in Main branch]
Normal read requests have this behavior, but size requests were not respecting this, which caused problems when the constructor launches a callback on another thread and then immediately calls WaitCompletion, such as in the callback lambda inside FFileCacheHandle::FFileCacheHandle / PushCompletedRequest.
#rb rune.stubbe
#jira none
#rnx
#ROBOMERGE-SOURCE: CL 12974187 in //UE4/Release-4.25/... via CL 12974195 via CL 12974203
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v682-12900288)
[CL 12974208 by jeff newquist in Main branch]
[RN] Minor, PakFiles
#ROBOMERGE-SOURCE: CL 12502978 via CL 12502983 via CL 12502984
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v673-12478461)
[CL 12502987 by matt peters in Main branch]
#rb steve.robb
#ROBOMERGE-SOURCE: CL 12450992 via CL 12450999 via CL 12457993
#ROBOMERGE-BOT: (v672-12450963)
[CL 12458149 by johan torp in Release-Engine-Staging branch]
[FYI] johan.torp, steve.robb, eric.knapik
#rnx
#ROBOMERGE-SOURCE: CL 12434782 via CL 12434783 via CL 12437070
#ROBOMERGE-BOT: (v671-12333473)
[CL 12443982 by david hamm in Release-Engine-Staging branch]
#rb steve.robb, eric.knapik
#ROBOMERGE-SOURCE: CL 12371098 via CL 12372591 via CL 12388959
#ROBOMERGE-BOT: (v671-12333473)
[CL 12390565 by johan torp in Release-Engine-Staging branch]
Legacy pak file versions encode pak entries in FPakFile::LoadLegacyIndex. That function repeatedly serializes bytes from the pakfile on disk into a reused FPakEntry. Serialize expects the FPakEntry to be in an empty constructed state, and it was not empty due to serializing previous PakEntries into it.
Added a reset function and call it in this one place where FPakEntry is reused.
#rb Graeme.Thornton
#rnx
[CL 12000363 by Matt Peters in Main branch]
#rb none
#testing iOS Client
#ROBOMERGE-SOURCE: CL 11843182 via CL 11843195 via CL 11843203
#ROBOMERGE-BOT: (v656-11643781)
[CL 11879820 by thomas ross in Main branch]
This was originally fixed in CL 11458603 but it did not fix the issue entirely and in any case I was not happy with the manual reset event approach.
The problem with the manual reset event was that there is no actual safe point at which we can reset the event without risking some requesting thread missing it and waiting forever, it is just a lot rarer than the original thread lock.
- Each FChunkRequest now can be given an event that is triggered, so the correct thread is always woken up if waiting.
- Since the signals are 1-1 we no longer risk having threads never being woken.
- We only wake the thread that has a request to process so don't risk flooding the cpu.
- Removed FChunkCacheWorker::SetupDecryptionKey as it was not implemented/used.
- Removed FChunkCacheWorker::ChunkRequestAvailable and its associated methods ::WaitForNextChunk/::FlushRemainingChunkCompletionEvents.
In testing this version runs slightly faster on loading tests (0.2seconds on average), although just removing the events entirely and running a busy loop with a yield ran slightly faster on machines with less cores (4 cores with hyperthreading or less) albiet with a much higher variation and so potentially could end up a lot worse.
#rb Graeme.Thornton
#jira UE-89165
#ushell-cherrypick of 11724338 by paul.chipchase
#ROBOMERGE-SOURCE: CL 11724753 in //UE4/Release-4.25/... via CL 11724760
#ROBOMERGE-BOT: RELEASE (Release-4.25Plus -> Main) (v656-11643781)
[CL 11724764 by paul chipchase in Main branch]
Fix incorrect use of static constexpr without inline.
#rb trivial
#rnx
#ROBOMERGE-SOURCE: CL 11465526 via CL 11465539
#ROBOMERGE-BOT: (v654-11333218)
[CL 11465545 by matt peters in Main branch]
FChunkCacheWorker::ChunkRequestAvailable could have many threads waiting on it and would only wake up a single thread each time a FChunkBuffer is processed. There would be no guarantee that the correct thread would be woken up or that each thread would be woken a single time, meaning that sooner or later we would end up with the FChunkCacheWorker having no more work to process but a number of threads still idle, waiting on the event to wake them, which would never happen.
Switching to a manually reset event will cause all waiting threads to be woken each time the event is triggered and will make sure that we don't deadlock.
This issue showed up now due to changes else where in the code which resulted in more threads waiting on FChunkCacheWorker than we previously had.
#jira UE-88734
#rb graeme.thornton
#ROBOMERGE-SOURCE: CL 11458603 in //UE4/Release-4.25/... via CL 11458605
#ROBOMERGE-BOT: RELEASE (Release-4.25Plus -> Main) (v654-11333218)
[CL 11462181 by paul chipchase in Main branch]
FChunkCacheWorker::ChunkRequestAvailable could have many threads waiting on it and would only wake up a single thread each time a FChunkBuffer is processed. There would be no guarantee that the correct thread would be woken up or that each thread would be woken a single time, meaning that sooner or later we would end up with the FChunkCacheWorker having no more work to process but a number of threads still idle, waiting on the event to wake them, which would never happen.
Switching to a manually reset event will cause all waiting threads to be woken each time the event is triggered and will make sure that we don't deadlock.
This issue showed up now due to changes else where in the code which resulted in more threads waiting on FChunkCacheWorker than we previously had.
#jira UE-88734
#rb graeme.thornton
#ROBOMERGE-SOURCE: CL 11458603 in //UE4/Release-4.25/...
#ROBOMERGE-BOT: RELEASE (Release-4.25 -> Release-4.25Plus) (v654-11333218)
[CL 11458605 by paul chipchase in 4.25-Plus branch]
#ROBOMERGE-SOURCE: CL 11458135 via CL 11458136 via CL 11458137 via CL 11458138
#ROBOMERGE-BOT: (v654-11333218)
[CL 11458139 by matt peters in Main branch]
OptimizedPakIndex: Bump the PakIndex version from 9 to 10, to avoid conflict with PakFile_Version_FrozenIndex = 9 from mainline. This change will remain in Fortnite only, and unlike UE::Main will refuse to load version 9 paks. This change can be clobbered with UE::Main's version after a few days when we have deleted all of our version 9 paks.
#ROBOMERGE-SOURCE: CL 11458117 via CL 11458118 via CL 11458119 via CL 11458120
#ROBOMERGE-BOT: (v654-11333218)
[CL 11458121 by matt peters in Main branch]
[REVIEW]
#rb Graeme.Thornton
Optimize Pak file indices by computing the PathHashIndex in unrealpak rather than using strings at runtime.
This also removes the need to shrink the string indices for performance, which was triggering a bug due to unsynchronized multithreaded access.
#ROBOMERGE-SOURCE: CL 11442058 in //Fortnite/Release-12.00/... via CL 11442063 via CL 11442073 via CL 11442081
[CL 11443295 by Matt Peters in Main branch]
#rb steve.robb
#ROBOMERGE-SOURCE: CL 11303014 via CL 11303031 via CL 11303037 via CL 11303042
#ROBOMERGE-BOT: (v0-11244347)
[CL 11303201 by graeme thornton in Main branch]
