#UE4 Added an extra canary check
--------
Integrated using branch Ue4-To-UE4-Fortnite-Simple (reversed) of change#2527111 by Bob.Tellez on 2015/04/27 12:42:15.
[CL 2527113 by Bob Tellez in Main branch]
#UE4 Added ShaderMagic_CleaningUp to the Canary checks in FShader. This will help track down the cause of UE-14017
--------
Integrated using branch Ue4-To-UE4-Fortnite-Simple (reversed) of change#2524950 by Bob.Tellez on 2015/04/24 16:48:53.
[CL 2527112 by Bob Tellez in Main branch]
- Linkers are no longer UObjects. Renamed ULinker, ULinkerLoad and ULinkerSave to FLinker, FLinkerLoad, FLinkerSave respectively
- Linkers are now associated with their UPackages
- Linker version is now stored in UPackages
- Async loading is now performed on a separate thread (if platform supports it and only in cooked builds), with the exception of PostLoad which is still done on the game thread
- Added UObject::IsPostLoadThreadSafe() function to determine if PostLoad is thread safe and can be executed on the async loading thread (defaults to false)
- UObject creation is now thread safe and can be performed on any thread
- Move many of the linker/UObject globals into FUObjectThreadContext (TLS)
- GetAsyncLoadPercentage() now takes PostLoad into account
- More async loading stats
- Added AtomicallySetFlags/ClearFlags to UObject
- Made FModuleManager thread safe.
- Added FGCScopeGuard as means of preventing GC from executing from non-game thread
- It's possible to disable async loading thread through ini settings.
- Cancelling async loading will now also trigger GC
- Implemented a basic version of async streaming priorities.
Change 2410813 by Mikolaj Sieluzycki:
Change Sleep in while loop to ConditionalSleep in FMultiReaderSingleWriterGT
Change 2410734 by Mikolaj Sieluzycki:
Make FModuleManager thread safe.
Change 2399879 by Mikolaj Sieluzycki:
Basic version of async streaming priorities.
Change 2410707 by Mikolaj Sieluzycki:
Implement conditional and no stat versions of sleep.
Change 2371939 by Robert Manuszewski:
Async Loading Improvements: adding more stats (accumulators)
Change 2372403 by Robert Manuszewski:
Fixing compile errors when STATs are not enabled
Change 2371526 by Robert Manuszewski:
AsyncLoading Improvements (WIP)
Change 2407198 by Robert Manuszewski:
Re-implementing delegate fixes for Async Loading
Change 2407425 by Robert Manuszewski:
Re-implementing cancelling async loading in the async loading branch.
Change 2484362 by Robert Manuszewski:
Making it possible to disable async loading thread through ini settings.
Change 2484744 by Robert Manuszewski:
Minimizing locks in GC and other threads when handling UObjects
Change 2480190 by Robert Manuszewski:
Fixing infinite stall after canceling async loading in non-cooked builds
Change 2484268 by Robert Manuszewski:
Fixing crash when allocating permanent object pool.
Change 2489761 by Robert Manuszewski:
Fixing BulkData using linker archive on the main thread even if the linker was created on the async loading thread.
Change 2493624 by Robert Manuszewski:
Cancelling async loading will now also trigger GC
Change 2487881 by Robert Manuszewski:
Making ShaderIdMap operations thread safe.
Change 2488067 by Robert Manuszewski:
Fixing GetAsyncLoadPercentage. It will now also respect PostLoad.
Change 2458640 by Robert Manuszewski:
Fixing crash in PIE
Change 2458825 by Robert Manuszewski:
Fixing a few crashes when streaming and the package is missing.
Change 2476935 by Robert Manuszewski:
Fixing crash while async loading ANavigationData
Change 2477361 by Robert Manuszewski:
Fixing crashes in cooked game
Change 2480095 by Robert Manuszewski:
Making FUObjectArray more thread safe
Change 2475443 by Robert Manuszewski:
Re-enabling single-threaded async loading path for the editor and platforms that don't support multithreading.
Change 2475458 by Robert Manuszewski:
Making sure bulk data is only loaded on a separate thread if it's not being loaded on the async loading thread.
Change 2476661 by Robert Manuszewski:
Fixing FlushAsyncLoading not flushing everything
Change 2401089 by Jaroslaw Surowiec:
Core - Added AtomicallySetFlags/ClearFlags to UObject, added a comment to ThisThreadAtomicallyClearedRFUnreachable
[CL 2498249 by Robert Manuszewski in Main branch]
- This is especially important on OS X OpenGL where these hitches can be equivalent to several frames, so it is enabled by default for this platform.
- The logging of draw calls is turned on/off with r.UseShaderDrawLog and incurs a fixed overhead as all RHI state is tracked & recorded into the shader cache.
- The actual predrawing is turned on/off with r.UseShaderPredraw and requires an existing cache to be useful, the predrawing takes place at frame-end and can be controlled with r.PredrawBatchTime which the time in ms to spend predrawing each frame until complete, or -1 to do it in one go (currently the default).
- The shader cache is now also versioned with the game able to supply a version distinct from the engine, so that old caches may be invalidated if/when required.
- The shader cache is now compatible with OpenGL SM4 & SM5 on Windows & probably Linux too.
- It may not work correctly with desktop ES2 emulation as that is still untested.
- To make it easier to add to other RHIs the API has been made simper to avoid lots of exposed branches.
#codereview michael.trepka, dmitry.rekman
[CL 2460109 by Mark Satterthwaite in Main branch]
* GetStreamOutElements is a callback function on the shadertype, just like ModifyCompilationEnvironment
* If GetStreamOutElements is specified, the FShaderResource includes that shader type in its ID as it can't be shared with any other shader type.
* FShader is no longer a FRenderResource which solves some multiple initialization issues
#codereview Marcus.Wassmer
[CL 2430024 by Daniel Wright in Main branch]
* Disabled by default, need a movable skylight with r.DistanceFieldGI enabled at load time (ConsoleVariables.ini)
* Surfels (oriented disks) are placed on mesh surfaces and capture diffuse color by evaluating the material with actual UVs and world position
* Surfels are lit and shadowed by ray tracing through mesh distance fields (directional light only so far)
* Irradiance cache records are lit with nearby Virtual Point Lights created from lit surfels. Only using camera based LOD of the surfel representation and object based surfel culling, no hierarchy yet.
* Shadowing of the VPLs is provided by distance field cone tracing. A depth visibility function is constructed along each of the 9 cone trace directions, and used to shadow the incoming VPL lighting.
* Component reregistration and transform updating causes surfels to be generated again which can be slow. In the future only the material needs to be re-evaluated.
* Geometry shaders with stream out are now supported, by implementing FShader::GetStreamOutElements
* Compute shaders can now evaluate materal attributes like BaseColor
* SkyLight MaxOcclusionDistance increased to 10m, which is the minimum useful amount for GI
[CL 2424208 by Daniel Wright in Main branch]
- The consolve variable "r.UseShaderCaching" controls whether the cache should be used.
- The cache has to be initialised and shutdown by the RHI.
- All RHI shader types now contain the SHA hash of the compiled source when the cache is in use so that they can be identified efficiently.
- All shaders are fetched from the cache rather than being created on-demand.
- On shader deserialisation the new shader is sent to the render thread for creation & caching.
- After each shader is cached all complete bound-shader-states that include that shader are constructed and cached.
- Wrapped OpenGL glCreateShader & glCreateProgram to use name-caching as these operations can synchronise the entire OpenGL pipeline on OS X.
[CL 2423996 by Mark Satterthwaite in Main branch]
This is controlled by the macro "WITH_CASE_PRESERVING_NAME", which is currently just set to "WITH_EDITORONLY_DATA" so that it works in editor builds (and UHT).
Added an extra NAME_INDEX entry to FName to store a second string table index for a case-variant string. The previous Index value (now called ComparisonIndex) is still used for comparison purposes (as FNames are still case-insensitive).
The Init process for an FName now works like this:
1) It will find or add a string table entry for the given string (not matching case) - this entry index is stored in ComparisonIndex.
2) It will then compare the string table entry string against the given string (matching case) to see if it also needs to add a case-variant entry for the FName.
3) If it does, it finds or adds a second string table entry (matching case) for the string - this entry index is stored in DisplayIndex.
Hard-coded FNames (those listed in UnrealNames.h) do not support case-variants (due to existing network replication rules for hard-coded FNames), so they skip steps 2 and 3.
I added FMinimalName, which is the same size as FName was previously. This shouldn't really be used (and as such, is deliberately awkward to make/use) as it loses the case-preserving behaviour of FName, however it was required for some things (like stats) that had a hard-coded upper limit on FName size.
I added FScriptName, which always contains the extra display index (even when WITH_CASE_PRESERVING_NAME is disabled). This is used by Blueprint bytecode, as the types used by Blueprint bytecode must be a consistent size between all build configurations.
Other changes:
- Fixed up any places that were passing an Index into the FName constructor which was supposed to take an EName.
- Some places were doing this to make the number unique when replicating an object, but this was losing the case-variant information, so I had to fix them.
- FName will now assert if the EName constructor is used with an value outside the range of hard-coded FNames.
- Ensured that assets, actors, and blueprint components could all be renamed in a way that only changed their case, and that these changes were correctly persisted.
- Added FLinkerNamePairKeyFuncs and TLinkerNameMapKeyFuncs for use with TSet and TMap.
- These allow ULinkerSave and ULinkerLoad to correctly write out case-variants for FNames, and also fixes an issue where the linker would erroneously write out duplicate string table entries for FNames which had a different number (causing package bloat).
- Bumped VER_MIN_SCRIPTVM_UE4 so that all Blueprint bytecode is recompiled using FScriptName.
ReviewedBy Robert.Manuszewski, Gil.Gribb
[CL 2300730 by Jamie Dale in Main branch]
- Store global bitfield of active feature levels
- Generate shaders for all active feature levels
- Add option to experimental settings to enable dynamic feature level
- Redirect GRHIFeatureLevel to GMaxRHIFeatureLevel - Effectively const now, ready to be removed
[CL 2296373 by Graeme Thornton in Main branch]
- Removed default shader platform parameter from GetGlobalShaderMap()
- Added an inline overload of GetGlobalShaderMap() that takes a feature level instead, and translates to the correct shader platform
- Cached feature level on FSceneView for faster access
- Cached shader map on FViewInfo for faster access
- Cached featurelevel/shadermap on rendering composition graph execution context, for faster access
Fixed a couple of crashes when switching feature level dynamically
- Needed to allow certain shader permutations to be generated if feature level was ES2, but still on a PC platform. (i.e. hitproxy shaders, and basepass shaders used by editor primitives system)
#codereview Nick.Penwarden
[CL 2275937 by Graeme Thornton in Main branch]
* Both paths still exist in code during this transition. The GetDynamicMeshElements path is currently disabled, enable with 'r.UseGetDynamicMeshElements 1'.
* DrawDynamicElements was called once per pass, all sorts of pass-specific cruft had leaked in along with short-term modifications of shared structures (vertex factories and material proxies). DrawDynamicElements was not const, so a bunch of 'update' type work had leaked in. Various PreRenderView implementations relied on the view being in the view family.
* GetDynamicMeshElements is called once in InitViews, and the resulting meshes are used in various passes. GetDynamicMeshElements is const and must not modify the proxy, update work should be pushed to the proxy from the game thread. GetDynamicMeshElements can be called on views that are not in the view family, which allows a proper view for shadow depth passes.
* View mode override functionality moved out of the individual proxies into a centralized place, ApplyViewModeOverrides
* GPU particles not working yet, FRecastRenderingSceneProxy not implemented
[CL 2253795 by Daniel Wright in Main branch]
