this is needed to fix up physx compiler errors
#rb none
#code.review ori.cohen
#lockdown josh.adams
#jira UE-36562
[CL 3151133 by Nick Shin in Main branch]
Move the Apple Driver Monitor stats into their own stat groups, DriverMonitor has the common stats, DriverMonitorAMD/Intel/Nvidia have the vendor/GPU specific stats. The Metal and OpenGL RHIs update the driver monitor stats group for the current GPU at the appropriate time.
Change 2898950 on 2016/03/08 by Mark.Satterthwaite
More shader cache code documentation.
Change 2898952 on 2016/03/08 by Michael.Trepka
Check GPU driver version and warn of bad drivers only on Windows
Change 2898964 on 2016/03/08 by Mark.Satterthwaite
Only verify the vertex attribute layout for Metal in debug builds or when using development with the debug layer turned on. It reduces performance significantly and isn't all that helpful unless you are attempting to debug a mismatch.
Change 2898973 on 2016/03/08 by Mark.Satterthwaite
Switch uniform buffers to managed memory on Mac as this is more appropriate for AMD & Nvidia GPUs.
Change 2898988 on 2016/03/08 by Mark.Satterthwaite
Simplify MetalContext by having only one SubmitCommandsHint implementation.
Change 2899011 on 2016/03/08 by Mark.Satterthwaite
Duplicate 4.11 CL #2898988:
Proper fix for UE-25804 - we have to manually expand PF_G8 + SRGB to RGBA8_sRGB - this then fixes UE-27483.
#jira UE-25804
#jira UE-27483
Change 2899024 on 2016/03/08 by Mark.Satterthwaite
Duplicate 4.11 CL #2887365 & CL #2887583:
Allow InfiltratorDemoEditor under Metal to issue a query buffer reset without crashing - the function that switches to the new query buffer needs to reapply some of the draw-state so that future commands don't dereference nil.
#jira UE-27513
My earlier fix for UE-27513 overlooked various internal details that meant it wouldn't restore state correctly, would fail validation and could crash in a new place. This version will ensure that cached state is only reset when it is appropriate to do so and will restore it correct when doing a query buffer reset.
#jira UE-27513
Change 2899418 on 2016/03/08 by Daniel.Lamb
Added support for textboxes in the editor to convert uasset filenames into long package names. As this is more useful to the cooker and more portable for projects.
#codereview Matt.Kuhlenschmidt
#jira UE-27785
Change 2899419 on 2016/03/08 by Daniel.Lamb
Added support for passing -opengl command through to launch on if the editor is started with it.
#codereview Michael.Trepka
Change 2900846 on 2016/03/09 by Mark.Satterthwaite
Reimplement Metal object lifetime tracking as stats in the stat-group, though the old system is maintained as a debug-only tool that could (and probably should) be extended to track over/under-release bugs. Currently the texture count will be distorted by texture SRVs so will need improvement but other stats should be reliable. In order to properly report the number of buffers the TResourcePool policy class must now define a FreeResource function, so I've added them to the appropriate places too.
Change 2900853 on 2016/03/09 by Mark.Satterthwaite
Optimise away empty encoders that don't perform a clear operation on AMD & Intel, but not Nvidia or non-Mac Metal devices. This should slightly improve performance.
Change 2900927 on 2016/03/09 by Mark.Satterthwaite
Implemented operation threshold submission of Metal command buffers to keep the GPU busier and not just idle waiting for the CPU. Whenever rhi.Metal.CommandBufferCommitThreshold is set to a value >0 and the current command buffer has >= draw/dispatch operations outstanding then the command-buffer will be committed at the next encoder boundary. The default value is 100 operations which is currently arbitrary and the feature can be disabled by setting the value to <= 0 in which case only explicit submissions will occur as previously.
Change 2901310 on 2016/03/09 by Mark.Satterthwaite
Change OneColor clear shader setup so that it works with parallel encoding in Metal.
Change 2903002 on 2016/03/10 by Mark.Satterthwaite
Instantiate the OneColor shaders once in Metal.
Change 2903274 on 2016/03/10 by Mark.Satterthwaite
Remove more unnecessary parallel execution stalls from MetalRHI.
Change 2903402 on 2016/03/10 by Mark.Satterthwaite
Implement Metal support for index buffer SRVs.
Change 2903419 on 2016/03/10 by Mark.Satterthwaite
Always use Managed memory on Mac Metal for buffers.
Change 2905206 on 2016/03/11 by Mark.Satterthwaite
Worked around UE-27818 "ElementalDemo Causes Invalid Rendering on AMD GPUs" - recent changes to allow mesh particles to write to velocity leave a texture-buffer unbound & then use a uniform value & an if-branch to guard against access but AMD's Mac GL driver notices that the buffer is referenced in the shader but not bound & promptly tries to fallback to Apple's S/W renderer regardless of what the uniform value is. That's legal behaviour for an OpenGL implementation so the C++ code has been changed to allocate and write the current transforms into the buffer for OpenGL when they wouldn't otherwise be provided. This is sufficient to avoid the problem without affecting any other API.
Change 2906217 on 2016/03/11 by Nick.Shin
re-enabled http network file server
it was disabled in CL: #2790193
#jira UE-22166 HTML5 Cook on the fly will launch and then close browser
Change 2908203 on 2016/03/14 by Michael.Trepka
Merging //UE4/Dev-Main to Dev-Platform (//UE4/Dev-Platform). Everything but SSF lib.
Change 2908553 on 2016/03/14 by Mark.Satterthwaite
Force a submit & wait in Metal when contexts are being destroyed to prevent kernel panics in drivers which continue to process the now abandoned command-queue and encounter invalid resources (because we destroy them on shutdown).
Change 2908595 on 2016/03/14 by Michael.Trepka
Fixed iOS compile error in MetalUniformBuffer.cpp
#codereview Mark.Satterthwaite
Change 2910106 on 2016/03/15 by Mark.Satterthwaite
Use a dispatch_semaphore not an FEvent for Metal free-list synchronisation as the dispatch_worker threads can't be properly setup for FStats and this causes problems.
Change 2910107 on 2016/03/15 by Mark.Satterthwaite
Fix Metal reporting of GPU memory through the RHI as it is in bytes, not MB.
Change 2910138 on 2016/03/15 by Mark.Satterthwaite
Properly retain/release dispatch_semaphore for Metal command buffer completion block & allow uniform buffer creation on parallel encoding thread.
Change 2911735 on 2016/03/16 by Nick.Shin
housekeeping
removing extra and inconsistant whitespace as well as making tabs & spaces consistant
[CL 2936662 by Josh Adams in Main branch]
and unwinding the previous checkin
#jira UE-25822 //UE4/Main: UE4_HTML5_Mono_Precompiled Warnings
#lockdown marc.audy
[CL 2849957 by Nick Shin in Main branch]
SetUpEmscriptenConfigFile() also updated to be generated via emscripten itself instead of manually writing it out
#jira UE-25822 -- //UE4/Main: UE4_HTML5_Mono_Precompiled Warnings
#lockdown josh.adams
[CL 2847435 by Nick Shin in Main branch]
Change 2783376 on 2015/11/30 by Nick.Shin
upgrading emscripten SDK to 1.35.9
following instruction from the README file
Change 2787414 on 2015/12/02 by Nick.Shin
upgrading emscripten to 1.35.0
removing old SDK and tools for Mac and Win64
Change 2790218 on 2015/12/04 by Nick.Shin
merge (CL: #2790164) from //UE4/Dev-Physics to //UE4/Dev-Platform
PhysX HTML5 bc files
Change 2794786 on 2015/12/08 by Nick.Shin
merge CL #2794757 part 1 of 2
from //UE4/Dev-Physics/PhysX/PhysX_3.3/Lib/html5
to //UE4/Dev-Platform/Engine/Source/ThirdParty/PhysX/PhysX-3.3/lib/HTML5/
Change 2794789 on 2015/12/08 by Nick.Shin
merge CL #2794758 part 2 of 2
from //UE4/Dev-Physics/PhysX/PhysX_3.3/Lib/html5
to //UE4/Dev-Platform/Engine/Source/ThirdParty/PhysX/PhysX-3.3/lib/HTML5/
Change 2799151 on 2015/12/10 by Dmitry.Rekman
Guarantee XGE.xml sorting order for 10+ builds.
- A licensee pointed out the problem that AutomationTool.UE4Build.FindXGEFiles() sorts the files by filename, so e.g. UBTExport.10.xge.xml takes priority over UBTExport.2.xge.xml.
#codereview Ben.Marsh
Change 2799440 on 2015/12/11 by Mark.Satterthwaite
Duplicate CL #2790251: Temporarily revert some of the changes for Mac mouse cursor locking as they were causing more problems than they solved.
Change 2799441 on 2015/12/11 by Mark.Satterthwaite
Duplicate CL #2796111 & #2796158: Fix cooking shader cache files - it wasn't being enabled despite a cached shader format being listed.
Change 2799442 on 2015/12/11 by Mark.Satterthwaite
Duplicate CL #2797758: Defer calls to AUGraphUpdate into FCoreAudioDevice::UpdateHardware - this call will synchronise the calling thread with the CoreAudio thread/run-loop so that the CoreAudio graph is safe to modify and this may incur a significant stall. This means it is far more efficient to amortise the cost of all changes to the graph with a single call. To ensure correctness the audio format conversion components are cached and disposed of after the call to AUGraphUpdate so that any existing operations on the CoreAudio thread are completed prior to disposal.
Change 2799601 on 2015/12/11 by Mark.Satterthwaite
Implement background reading of NSPipe's in Mac ExecProcess to avoid the sub-process blocking trying to write to the meagre 8kb internal buffers. This may fix problems with SVN on Mac.
Change 2799657 on 2015/12/11 by Mark.Satterthwaite
Remove the hlslcc major version from the Metal and OpenGL shader formats to ensure that there are enough bits to represent the different version components. There's no expectation that the major version of hlslcc will change and it will soon be removed entirely.
Change 2799691 on 2015/12/11 by Mark.Satterthwaite
Merging final internal-only changes from WWDC.
Change 2800182 on 2015/12/11 by Mark.Satterthwaite
Capture the system.log contents from the moment we boot to the point we crash to report GPU restarts and other system errors not written into our own logs.
Change 2801395 on 2015/12/14 by Mark.Satterthwaite
Fix the Metal shader compiler so that it properly reports the number of sampler objects in use, not the number of textures as Metal separates its 16 samplers and up-to 128 textures in a single shader stage, like D3D and unlike OpenGL. This fixes a lot of material compile errors in newer projects which aren't being designed for obsolete OpenGL.
Change 2801653 on 2015/12/14 by Daniel.Lamb
Load package differ can now diff header part of packages.
Changed the way IsChildCooker is handled improves performance of multiprocess cooker.
Change 2801655 on 2015/12/14 by Daniel.Lamb
Added cooker warning to resave packages if they don't have collision data for their static meshes.
Added NavCollision creation on static mesh import so that we save out the NavCollision.
Change 2801923 on 2015/12/14 by Daniel.Lamb
Fix compilation error with CreateLoader.
Change 2802076 on 2015/12/14 by Daniel.Lamb
Remove some debugging assistance code.
Change 2803207 on 2015/12/15 by Mark.Satterthwaite
Add missing Metal formats for PF_R16_SINT/UINT.
Change 2803254 on 2015/12/15 by Mark.Satterthwaite
Add additional uint/2/3/4 overrides for SV_Target(x) to MetalUtils and when generating the output variable look for an exact type match before restoring to the first match with the correct number of elements. This ensures that we generate uint/2/3/4 writes when required for CopyStencilToLightingChannelsPS without breaking anything else.
Change 2803259 on 2015/12/15 by Mark.Satterthwaite
Fix stencil texture swizzle for Metal which uses .x not .g for stencil value.
Change 2803262 on 2015/12/15 by Mark.Satterthwaite
Fix FMetalRHICommandContext::RHISetScissorRect handling 0 sized rects when RHISetScissorRect is called before RHISetViewport.
Change 2803321 on 2015/12/15 by Mark.Satterthwaite
Duplicate CL #2786291: Fix Metal validation errors caused by incorrect instance count and also a crash-bug caused by accessing a defunct depth-stencil texture. This should be enough to ensure Metal works even if you've been playing previously with OpenGL.
Change 2803413 on 2015/12/15 by Mark.Satterthwaite
Workaround the Material Editor's unfortunate habit of rendering tiles without a depth/stencil-buffer attached despite tiles wanting to write to depth - in Metal we have to create a temporary Depth-Stencil texture so that we don't crash the driver because it won't rewrite the shaders for us (unlike D3D/GL).
Change 2806247 on 2015/12/16 by Daniel.Lamb
Fixed UParticleRequiredModule deterministic cook issue.
#codereview Olaf.Piesche
Change 2806834 on 2015/12/17 by Mark.Satterthwaite
Temporarily work around absence of Checked & Shipping APEX/PhysX binaries on Mac.
Change 2807017 on 2015/12/17 by Mark.Satterthwaite
Handle the shader cache being initialised for cooking multiple times until I can sort out the implementation properly.
Change 2807027 on 2015/12/17 by Daniel.Lamb
Enabled DDC stats.
==========================
MAJOR FEATURES + CHANGES
==========================
Change 2719147 on 2015/10/07 by Mark.Satterthwaite
Allow the shader cache to perform some precompilation synchronously on load before falling back to asynchronous compilation to balance load times against total time spent precompiling. Added a stat to the group that reports how long the precompile has been running until it completes so it is easier to track.
Change 2719182 on 2015/10/07 by Mark.Satterthwaite
Refactor the ShaderCache's internal data structures and change the way we handle recording whether a particular predraw state has been submitted to try and make it more efficient.
Change 2719185 on 2015/10/07 by Mark.Satterthwaite
Merging CL #2717701: Try and fix random crashes on Mac when manipulating bound-shader-states caused by ShaderCache potentially providing a bogus shader state pointer on exit from predraw.
Change 2719434 on 2015/10/07 by Mark.Satterthwaite
Make sure that Mac ensures reports have a source context and a sane callstack when sent to the crash-reports server.
Change 2724764 on 2015/10/12 by Josh.Adams
[Initial AppleTV support]
Merging //depot/YakBranch/... to //UE4/Dev-Platform/...
Change 2726266 on 2015/10/13 by Lee.Clark
PS4 - Calc reserve size required for DMA copy when using unsafe command buffers
Change 2726401 on 2015/10/13 by Mark.Satterthwaite
Merging CL #2716418: Fix UE-15228 'Crash Report Client doesn't restart into project editor on Mac' by reporting the original command line supplied by LaunchMac, not the modified one that strips the project name. The CRC can then relaunch as expected.
#jira UE-15228
Change 2726421 on 2015/10/13 by Lee.Clark
PS4 - Don't try to clear invalid targets
Change 2727040 on 2015/10/13 by Michael.Trepka
Merging CL 2724777 - Fixed splash screen rendering for images with DPI different than 72
Change 2729783 on 2015/10/15 by Keith.Judge
Fix huge memory leak in Test/Shipping configurations, caused because I am a numpty.
Change 2729847 on 2015/10/15 by Mark.Satterthwaite
Merging CL #2729846: On OS X unconstrain windows from the dimension of the parent display when in Windowed mode - it is OK for them to be larger in this case. They do need to be repositioned if on the Primary display so that they don't creep under the menu bar and become unmovable/unclosable and Fullscreen windows still need to be constrained to a single display. We can now take screenshots of windows that are larger than the display & not get grey bars beyond the cutoff.
#jira UE-21992
Change 2729865 on 2015/10/15 by Keith.Judge
Fast semantics - Finish up resource transitions, adding resource decompression where appropriate and using non-fast clears where we can't determine the resource transition.
Change 2729897 on 2015/10/15 by Keith.Judge
Fast Semantics - Make sure all GetData() calls are made safe with GPU fences.
Change 2729972 on 2015/10/15 by Keith.Judge
Removed the last vestiges of ID3D11DeviceContext/ID3D11DeviceContext1 from the Xbox RHI. Everything now uses ID3D11DeviceContextX directly.
This should be marginally quicker as it stops a double call to ClearState().
Change 2731503 on 2015/10/16 by Keith.Judge
Added _XDK_VERSION to the DDC key for textures, which should solve the issue of the tiling mode changing in August XDK (and future changes Microsoft may inflict).
Change 2731596 on 2015/10/16 by Keith.Judge
Fast Semantics - Add deferred resource deletion queue to make deleted resources be actually deleted a number of frames later so that the GPU is definitely finished with them. Hooked up the temporary SRVs for dynamic VBs as a first step.
Change 2731928 on 2015/10/16 by Michael.Trepka
PR #1659: Mac/Build.sh handles additional arguments (Contributed by judgeaxl)
Change 2731934 on 2015/10/16 by Michael.Trepka
PR #1618: added clang 3.7.0 -Wshift-negative-value ignore in JpegImageWrapper.cpp (Contributed by bsekura)
Change 2732018 on 2015/10/16 by Mark.Satterthwaite
Emit a shader code cache for each platforms requested shader formats, this is separate to the targeted formats as not all can or need to be cached.
- The implementation extends the ShaderCache's hooks in FShaderResource's serialisation function to capture the required shaders.
- Each target platform has its own list of cached shader formats, analogous to the list of targeted RHIs. Presently only the Mac implements this.
- Code cached shaders are now compressed (for size) to reduce the overhead associated with keeping all the shader code around - this works esp. well for text-based formats like GLSL.
Change 2732365 on 2015/10/16 by Josh.Adams
- Packaging a TVOS .ipa now works (still haven't tried any of the Editor integration like Launch On)
Change 2733170 on 2015/10/18 by Terence.Burns
Fix for Android IAP query not returning entire inventory.
Change 2733174 on 2015/10/18 by Terence.Burns
Fix Movie player issue where wait for movie to finish isnt being respected.
Seems a stray bUserCanceled event flag was causing this not to be observed.
Added some verbose logging to apple movie player.
Change 2733488 on 2015/10/19 by Mark.Satterthwaite
Added the ability to merge the .ushadercache files used by the ShaderCache to store shader & draw state information.
- Fixed a bug that would cause invalid shader membership and draw state information to be logged.
- Added a separate command-line tool to merge shader cache files, currently Mac-only but in theory should work on other platforms too.
Change 2735226 on 2015/10/20 by Mark.Satterthwaite
Fix temporal AA rendering on GL/Mac OS X - you can't rely on EyeAdaptation values unless SM5 is available so only perform that code on SM5 & we must correctly clamp saturate(NaN) to 0 as the current hlslcc won't do that for us (& is required by the HLSL spec). The latter used to be clamped in the AA_ALPHA && AA_VELOCITY_WEIGHTING code block that was removed recently.
#jira UE-21214
#jira UE-19913
Change 2736722 on 2015/10/21 by Daniel.Lamb
Improved performance of cooking stats system.
Change 2737172 on 2015/10/21 by Daniel.Lamb
Improved cooking stats performance for ddc stats.
