- This allows for Confidential platforms to exist outside of the engine, and insert themselves in as needed
- Directory structure is, where .... mirrors the directory structure for Engine and projects
- /Platforms/XXX/....
- Moving to more data driven approach for ShaderPlatforms and PlatformInfo, where they can be read from DataDrivenPlatformInfo.ini files that live in the platform config folders
- Removed platform mentions from UBT, by way of changing some enums to partial classes with static members (see UnrealTargetPlatform)
- Various other UBT/UAT modifications to allow for looking in other locations for files
- THIS IS NOT A COMPLETE AND FINAL SOLUTION. WE WILL CONTINUE WORK IN DEV-BUILD BEFORE ITS READY FOR PRIMETIME
#rb ben.marsh
[CL 6271418 by Josh Adams in Dev-Build branch]
The same ubulk file, typically for a texture, was written by two different threads. This caused a fatal SavePackage assert in FAsyncWriteWorker::DoWork(). There was no need to write out these files to disk at all.
Also cleaned up surrounding code a little bit.
#jira ue-72635
#rb robert.manuszewski
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: johan.torp
#ROBOMERGE-SOURCE: CL 5937291 via CL 5937787 via CL 5937827
#ROBOMERGE-BOT: BUILD (Main -> Dev-Build)
[CL 5956288 by johan torp in Dev-Build branch]
The editor specific feature of querying the file system to identify newly created packages (not having a corresponding file on disk) is really slow.
Early out from this expensive check when the package already has a known file size.
VTune user-mode cpu sampling time results when cooking a big data set (~200 000 packages):
UCookOnTheFlyServer::MaybeMarkPackageAsAlreadyLoaded() down from 10 sec to 0.1 sec, a 100X speedup.
FLinkerLoad::LoadAllObjects down from 400 sec to 280 sec, a 1.4X speedup, i.e. ~4% (120 sec) of the total load time of 2750 sec.
#rb robert.manuszewski
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 5818305 via CL 5818901 via CL 5818980
#ROBOMERGE-BOT: BUILD (Main -> Dev-Build)
[CL 5850321 by pj kack in Dev-Build branch]
New AssetRegistryState::InitializeFromExistingAndPrune temporarily disabled until some bugs have been fixed.
Test Scenario:
1) BuildCookStageAndRun with these arguments: -platform=Win64 -configuration=Development
2) CookIterate with these arguments: -run=Cook -CookCultures=en -TargetPlatform=WindowsClient -unversioned -stdout -unattended -iterate
Wall Time Results (as an average of running step 2) two times):
Before: ~09:40 (580 seconds) cook commandlet time
After: ~02:30 (150 seconds) cook commandlet time
=> 07:10 (430 seconds ) faster, i.e. a ~ 3.9x speedup
Win32 FileSystem Results:
Before: 1.5 million GetFileAttribute calls and 1.2 million FindNextFile calls
After: 35 0000 GetFileAttribute calls and 1.6 million FindNextFile calls
=> ~400 000 calls to FindNextFile replaces ~1.5 million calls to GetFileAttribute
#rb none
(peafour-cherrypick of //UE4/Dev-Core/[at]5645695 by PJ.Kack)
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 5533504 via CL 5533655 via CL 5536177 via CL 5772728 via CL 5772753 via CL 5772793
#ROBOMERGE-BOT: BUILD (Main -> Dev-Build)
[CL 5783689 by pj kack in Dev-Build branch]
Second optimization to a heavily hit function that uses an FArchiveFromStructuredArchive adapter. That class needs optimization, so temporarily use a straight FArchive path when text archive support is compiled out
fyi ben.woodhouse, gil.gribb, per.larsson
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5716883 via CL 5716884 via CL 5716885 via CL 5716886 via CL 5717811 via CL 5717889
#ROBOMERGE-BOT: BUILD (Main -> Dev-Build)
[CL 5728173 by graeme thornton in Dev-Build branch]
#rb none
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 5707932 via CL 5708541 via CL 5708552 via CL 5708563 via CL 5708976 via CL 5709035
#ROBOMERGE-BOT: BUILD (Main -> Dev-Build)
[CL 5718534 by robert manuszewski in Dev-Build branch]
Integrate all cook -iterate optimizations from //UE4/Dev-core/...
Test Scenario:
1) BuildCookStageAndRun with these arguments: -platform=Win64 -configuration=Development
2) CookIterate with these arguments: -run=Cook -CookCultures=en -TargetPlatform=WindowsClient -unversioned -stdout -unattended -iterate
Wall Time Results (as an average of running step 2) two times):
Before: ~09:40 (580 seconds) cook commandlet time
After: ~02:30 (150 seconds) cook commandlet time
=> 07:10 (430 seconds ) faster, i.e. a ~ 3.9x speedup
Win32 FileSystem Results:
Before: 1.5 million GetFileAttribute calls and 1.2 million FindNextFile calls
After: 35 0000 GetFileAttribute calls and 1.6 million FindNextFile calls
=> ~400 000 calls to FindNextFile replaces ~1.5 million calls to GetFileAttribute
#rb none
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 5533504 via CL 5533655 via CL 5536177
#ROBOMERGE-BOT: BUILD (Main -> Dev-Build)
[CL 5544723 by pj kack in Dev-Build branch]
#jira
#rnx
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 5494625 via CL 5494635 via CL 5494639 via CL 5497857 via CL 5497991
#ROBOMERGE-BOT: BUILD (Main -> Dev-Build)
[CL 5512110 by marc audy in Dev-Build branch]
#rnx
#ROBOMERGE-OWNER: ben.marsh
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 5483896 via CL 5483909 via CL 5483913 via CL 5487278 via CL 5487454
#ROBOMERGE-BOT: BUILD (Main -> Dev-Build)
[CL 5507511 by marc audy in Dev-Build branch]
