#rb gil.gribb
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 6417518 via CL 6418865 via CL 6418880 via CL 6418961 via CL 6418984
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 6497723 by ben woodhouse in Dev-VR branch]
The same ubulk file, typically for a texture, was written by two different threads. This caused a fatal SavePackage assert in FAsyncWriteWorker::DoWork(). There was no need to write out these files to disk at all.
Also cleaned up surrounding code a little bit.
#jira ue-72635
#rb robert.manuszewski
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: johan.torp
#ROBOMERGE-SOURCE: CL 5937291 via CL 5937787 via CL 5937827
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 5956336 by johan torp in Dev-VR branch]
The editor specific feature of querying the file system to identify newly created packages (not having a corresponding file on disk) is really slow.
Early out from this expensive check when the package already has a known file size.
VTune user-mode cpu sampling time results when cooking a big data set (~200 000 packages):
UCookOnTheFlyServer::MaybeMarkPackageAsAlreadyLoaded() down from 10 sec to 0.1 sec, a 100X speedup.
FLinkerLoad::LoadAllObjects down from 400 sec to 280 sec, a 1.4X speedup, i.e. ~4% (120 sec) of the total load time of 2750 sec.
#rb robert.manuszewski
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 5818305 via CL 5818901 via CL 5818980
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 5856467 by pj kack in Dev-VR branch]
New AssetRegistryState::InitializeFromExistingAndPrune temporarily disabled until some bugs have been fixed.
Test Scenario:
1) BuildCookStageAndRun with these arguments: -platform=Win64 -configuration=Development
2) CookIterate with these arguments: -run=Cook -CookCultures=en -TargetPlatform=WindowsClient -unversioned -stdout -unattended -iterate
Wall Time Results (as an average of running step 2) two times):
Before: ~09:40 (580 seconds) cook commandlet time
After: ~02:30 (150 seconds) cook commandlet time
=> 07:10 (430 seconds ) faster, i.e. a ~ 3.9x speedup
Win32 FileSystem Results:
Before: 1.5 million GetFileAttribute calls and 1.2 million FindNextFile calls
After: 35 0000 GetFileAttribute calls and 1.6 million FindNextFile calls
=> ~400 000 calls to FindNextFile replaces ~1.5 million calls to GetFileAttribute
#rb none
(peafour-cherrypick of //UE4/Dev-Core/[at]5645695 by PJ.Kack)
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 5533504 via CL 5533655 via CL 5536177 via CL 5772728 via CL 5772753 via CL 5772793
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 5787307 by pj kack in Dev-VR branch]
Second optimization to a heavily hit function that uses an FArchiveFromStructuredArchive adapter. That class needs optimization, so temporarily use a straight FArchive path when text archive support is compiled out
fyi ben.woodhouse, gil.gribb, per.larsson
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5716883 via CL 5716884 via CL 5716885 via CL 5716886 via CL 5717811 via CL 5717889
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 5729182 by graeme thornton in Dev-VR branch]
#rb none
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 5707932 via CL 5708541 via CL 5708552 via CL 5708563 via CL 5708976 via CL 5709035
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 5718602 by robert manuszewski in Dev-VR branch]
Integrate all cook -iterate optimizations from //UE4/Dev-core/...
Test Scenario:
1) BuildCookStageAndRun with these arguments: -platform=Win64 -configuration=Development
2) CookIterate with these arguments: -run=Cook -CookCultures=en -TargetPlatform=WindowsClient -unversioned -stdout -unattended -iterate
Wall Time Results (as an average of running step 2) two times):
Before: ~09:40 (580 seconds) cook commandlet time
After: ~02:30 (150 seconds) cook commandlet time
=> 07:10 (430 seconds ) faster, i.e. a ~ 3.9x speedup
Win32 FileSystem Results:
Before: 1.5 million GetFileAttribute calls and 1.2 million FindNextFile calls
After: 35 0000 GetFileAttribute calls and 1.6 million FindNextFile calls
=> ~400 000 calls to FindNextFile replaces ~1.5 million calls to GetFileAttribute
#rb none
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 5533504 via CL 5533655 via CL 5536177
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 5546459 by pj kack in Dev-VR branch]
#jira
#rnx
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 5494625 via CL 5494635 via CL 5494639 via CL 5497857 via CL 5497991
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 5516123 by marc audy in Dev-VR branch]
#rnx
#ROBOMERGE-OWNER: ryan.vance
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 5483896 via CL 5483909 via CL 5483913 via CL 5487278 via CL 5487454
#ROBOMERGE-BOT: DEVVR (Main -> Dev-VR)
[CL 5510133 by marc audy in Dev-VR branch]
