GUBP High Level
* Temp Storage is zipped into a single archive per node now. This results in ~75% reduction in temp storage usage and network traffic, not to mention the per-file overhead.
* Temp Storage is in P:\\Builds\\{Game}\\TmpStore instead of P:\\Builds\\{Game}\\GUBP (to facilitate easier cleaning of this new structure).
* Temp Storage nodes are in subdirectories of {Branch}\\{CL}\\{NodeName} now instead of a flat directory structure that was hard to manually sift through.
GUBP Mid Level
* Removed -Store= and -StoreSuffix= test parameters.
* Added -NoZipTempStorage parameter to turn off temp storage zipping if necessary.
* Created GUBP.JobInfo class that collects info about the job as a whole to be passed around by GUBP. Mostly used by any code that need to interact with TempStorage.
* Created TempStorageNodeInfo that describes the necessary parameters to find the temp storage location for a node.
* Fully XML commented TempStorage.cs, and commented internals all major functions.
* Added a bunch of telemetry data for storing, retrieving, and cleaning shared temp storage.
UAT Mid Level
* Fixed a bug in Ionic.Zip that make ExtractAll() not work on Mono, checked in new DLLs.
* Added UAT parameter -UseLocalBuildStorage that allows you to test build storage stuff completely locally. Writes to Engine\\Saved\\LocalBuilds\\...
GUBP Low Level
* Refactored some GUBP startup code so temp vars would be limited in scope. Makes it easier to track the impact of refactoring these things.
* CullNodesForPreflight is only called for preflight builds.
* Refactored TempStorage.FindTempStorageManifests to use new TmpStore structure and harden the brittle string/path parsing it was doing. See the new TempStorage FindMatchingSharedTempStorageNodeCLs().
* Refactored TempStorage Saving and Loading to use XDocument instead of older XmlDocument. Removed a bunch of redundant checks.
* Use StripBaseDirectory and MakeRerootedFilePath to remove the brittle directory manipulation code. Directories no longer require a '/' at the end.
* Removed a few redundant caching layers in cleaning temp storage that try to ensure we don't clean a folder twice. None of them were necessary.
* Removed unused single-threaded copy code from temp storage.
* Updated Temp Storage unit test, and fully commented the logic behind it.
UAT Low Level
* UAT top level exception handler is now a single log line now to help parsers find the error.
* Removed several uses of FormatException as it doesn't display the entire exception chain, and is not as good as the default exception formatter.
* Removed ExceptionToString as it used FormatException, which was not a good precedent.
* Fixed several cases of exception propagation that was not properly chaining the inner exception.
* Refactored ThreadedCopyFiles to use Parallel.For because it was just as fast (if not faster) and much simpler to maintain.
* Removed the suffix from Robust_FileExists_NoExceptions because it's sole purpose in life WAS to throw exceptions!
* Added a bunch of XML doc comments to CommandUtils.
* Modernized some container manipulation and iteration to use IEnumerable and extension methods more appropriately.
* Added several @todos for other minor cleanup stuff that should happen eventually.
* Fixed some uses of String.Compare to use invariant culture.
#codereview:ben.marsh
[CL 2644846 by Wes Hunt in Main branch]
Previously, the file discovery phase of the asset gathering process would block the entire process until it had found all of the files it needed to test.
This can take quite some time for projects with a large amount of assets, so this change adjusts the process so that discovered paths will be reported back to the asset registry while the file discovery is on-going. This allows you to prioritize the scanning of certain folders before the initial discovery phase has finished.
To account for the fact that the asset registry will be reporting process of an unknown range, the asset discovery indicator has been updated to display a marquee bar until the initial file discovery has finished.
It will now show you the following messages:
- "Initializing Asset Discovery" - Loading the cached asset registry data and waiting for the file scan to start. The progress bar will marquee while this is happening.
- "Discovering Asset Files" - Shows an increasing count of the number of asset files that have been discovered on disk. The progress bar will marquee while this is happening.
- "Discovering Asset Data" - Shows a decreasing count of assets that need to have their asset data either read from disk, or copied from the cache file. The progress bar will show the percentage of the assets processed by the asset registry compared to the total number of assets to load and process.
- "Processing Asset Data" - Shows a decreasing count of assets that the asset registry still needs to process the gathered asset data for. The progress bar will show the percentage of the assets processed by the asset registry compared to the total number of assets to load and process.
One side effect of having the discovery phase scan go wide (rather than deep - to allow you to prioritize the scanning of certain folders) is that it may now report back folders that don't contain any assets. This essentially addresses a long time complaint from users, whereby the Content Browser wouldn't show empty folders.
This change also has the cached asset registry file write out for every 50,000 files that are read from disk. This avoids you needing to perform repeated work if you restart the editor before the entire scan has finished.
[CL 2644762 by Jamie Dale in Main branch]
[UE-18355] - Clicking on show viewport controls on mac comes up with missing page error on mac
#fix Prefixed URLs correctly with file:// in SLevelViewportControlsPopup
[CL 2644736 by Chris Wood in Main branch]
--Deleted a couple of helper functions that were used for this.
--Removed the old individual tests, replaced with one test.
--New tests will show in the UFE automation tab with the same naming scheme.
[CL 2644711 by Justin Hair in Main branch]
