Copying //UE4/Dev-VR to //UE4/Dev-Main (Source: //UE4/Dev-VR @ 4296730)

#lockdown Nick.Penwarden

============================
  MAJOR FEATURES & CHANGES
============================

Change 3775128 by Nick.Atamas

	Merging //UE4/Partner-Google-VR @ CL 3770408 to Dev-VR (//UE4/Dev-VR)

Change 4166311 by Jason.Bestimt

	#DEV-VR - Merge from //UE4/Dev-MagicLeap/... @ CL 4136411

Change 4185551 by Joe.Graf

	Added support to query and specify the desired video format for an AR session

Change 4269190 by Ethan.Geller

	[Dev-VR] #jira UE-62644 Deescalate resampling logs to display, and make sure that Lumin does not get Resampling settings from Android runtime settings.

Change 4269386 by Ryan.Vance

	#jira UE-62274

	We need to recompute the view frustum and related distances if the projection matrix is changed.

Change 4269401 by Ryan.Vance

	#jira UE-62155

	Debug canvas layer fixes

Change 4270710 by Chance.Ivey

	Merging //UE4/Dev-Partner-MagicLeap-4.20 to Dev-VR (//UE4/Dev-VR)
	 - 4262963
	 - 4261100
	 - 4261103
	 - 4265565

Change 4270822 by Ryan.Vance

	#jira UE-61462

	Work around for csm shadow culling bug on Lumiun

Change 4271396 by Jason.Bestimt

	#DEV-VR - Saving off original MLSDK variable for future editor "restarts" within the same process

	#JIRA: UE-58634

Change 4271402 by Jason.Bestimt

	#DEV-VR - Making bUseMobileRendering non-editable for binary builds

	#JIRA: UE-60956

Change 4271454 by Jason.Bestimt

	#DEV-VR - Integrating CL 4239939, 4236178 and 4269702 from Partners-Google-VR

Change 4272387 by Ryan.Vance

	#jira UEVR-1281

	Oculus 1.28 Plugin changes

Change 4273410 by Jason.Bestimt

	#DEV-VR - Fixing MagicLeapSDK Settings to be globaluserconfig (by removing the defaultconfig).

	#JIRA: UE-62092

Change 4273929 by Ryan.Vance

	#jira UE-62744

	Fixing missing includes..

Change 4274147 by Kyle.Fung

	Round-robin occlusion queries for stereoscopic rendering
	With the vr.RoundRobinOcclusion flag turned on, each frame will only render occlusion queries for one eye with an alternating scheme (ie odd frames only left eye, even frames only right eye).
	Additionally, the interface for class FPrimitiveOcclusionHistory has been changed along with a slight modification in its implementation. With round-robin occlusion, when extracing history entries we perform a search through the entire history to look for the oldest entry in a primitive's occlusion history.

	#jira UEVR-1289

Change 4274206 by Kyle.Fung

	Fixed some compiler warnings
	#jira UEVR-1289

Change 4275212 by Jason.Bestimt

	#DEV-VR - Fixing CIS error (forcing value to bool)

Change 4275973 by Jason.Bestimt

	#DEV-VR - Merging CL 4275879 from Partner-Google-VR

	GoogleVR 1.170 Release Update.
	- Removed reference to GetHMDDeviceType.
	- Updated to final version of SDK libraries.

Change 4276154 by Jason.Bestimt

	#DEV-VR - fix for CIS warning about PngPtr being used before it is checked

Change 4277171 by Jeff.Fisher

	UE-62399 Graphical corruption when opening the debug console
	-Switched from the fast hardware clear to the slower shader clear because the slower one works.  Perhaps a better fix exists that would make the faster clear work?  Costs 60us when debug ui is up, but only when rendering debug to a stereo layer (vr/ar).
	#jira UE-62399
	#review-4277026

Change 4277633 by Joe.Conley

	#jira UE-54565 - "Packaged game/Editor with Stereo Layers crashes when exiting game through steam VR overlay"

	Don't call GetStereoLayers if stereo is not enabled.

Change 4277723 by Ryan.Vance

	#jira UE-62776

	Cleaning up questionable partner code found by SA.
	I'm pretty confident the SA warning is a false positive, but there was a lot to fix here regardless.

Change 4280289 by Joe.Graf

	Added some "Ignored by ARCore" comments Google requested

Change 4280479 by Joe.Graf

	Fixed a member copy step in AppleARKitFrame::operator= that Wanghao reported

Change 4280824 by Jason.Bestimt

	#DEV-VR - removing stale comment about ML SDK Settings using defaultengine.ini

Change 4281117 by Jason.Bestimt

	#DEV-VR - Removing warnings on LUMIN from trying to load missing modules

	#JIRA: UE-62748

Change 4281514 by Jason.Bestimt

	#DEV-VR - Moving FunctionalTesting dependency under bCompileAgainstEngine

Change 4281847 by Joe.Graf

	Added a UPL file to AppleARKit that modifies the plist to include the requirement of ARKit and request for camera access

	#jira: UE-60050, UE-62837

Change 4281945 by Joe.Graf

	Removed the ensures from the ARBlueprintLibrary code since the code handles the failure cases properly

	#jira: UE-61845, UE-61846

Change 4282098 by Ethan.Geller

	#jira UE-62863 fix audio on lumin #rb none

Change 4283974 by Joe.Graf

	Deprecated bIsTracked on tracked images and face geo per Google API reunification feedback

Change 4284118 by Jason.Bestimt

	#DEV-VR - Moving FunctionalTest dependency to be both non-shipping and CompileAgainstEngine.  Allowing FunctionalTesting to be compiled in binary build

Change 4285339 by Ethan.Geller

	#jira UE-62087 Whitelist AudioCapture module for mac while not whitelisting the AudioCaptureEditor module, which should only work on Windows.

Change 4286024 by Ryan.Vance

	Copying //UE4/Partner-Microsoft-XR to Dev-VR (//UE4/Dev-VR)
	Windows mixed reality integration.
	Disabled by default

Change 4289812 by Jason.Bestimt

	#DEV-VR - Fix for non-unity build issue with windows header

Change 4296399 by Jason.Bestimt

	RSync fixes for xml files

	#JIRA: UE-62994

Change 4296679 by Jason.Bestimt

	#DEV-VR - Fixing remote iOS compile issue with BP projects that require plugins

	#JIRA: UE-62967

[CL 4299100 by Jason Bestimt in Main branch]

Engine/Source/Runtime/HeadMountedDisplay/Private/DefaultSpectatorScreenController.cpp

@@ -10,6 +10,7 @@
 #include "Engine/Texture.h"
 #include "HeadMountedDisplayBase.h"
 #include "SceneUtils.h" // for SCOPED_DRAW_EVENT()
+#include "IStereoLayers.h"
 
 
 FDefaultSpectatorScreenController::FDefaultSpectatorScreenController(FHeadMountedDisplayBase* InHMDDevice)
@@ -65,12 +66,16 @@ void FDefaultSpectatorScreenController::SetSpectatorScreenModeTexturePlusEyeLayo
 	}
 }
 
+void FDefaultSpectatorScreenController::QueueDebugCanvasLayerID(int32 LayerID)
+{
+	DebugCanvasLayerIDs.Add(LayerID);
+}
+
 FSpectatorScreenRenderDelegate* FDefaultSpectatorScreenController::GetSpectatorScreenRenderDelegate_RenderThread()
 {
 	return &SpectatorScreenDelegate_RenderThread;
 }
 
-
 struct FRHISetSpectatorScreenTexture final : public FRHICommand<FRHISetSpectatorScreenTexture>
 {
 	FDefaultSpectatorScreenController* SpectatorScreenController;
@@ -229,7 +234,7 @@ void FDefaultSpectatorScreenController::UpdateSpectatorScreenMode_RenderThread()
 	}
 }
 
-void FDefaultSpectatorScreenController::RenderSpectatorScreen_RenderThread(FRHICommandListImmediate& RHICmdList, FRHITexture2D* BackBuffer, FTexture2DRHIRef SrcTexture, FVector2D WindowSize) const
+void FDefaultSpectatorScreenController::RenderSpectatorScreen_RenderThread(FRHICommandListImmediate& RHICmdList, FRHITexture2D* BackBuffer, FTexture2DRHIRef SrcTexture, FVector2D WindowSize)
 {
 	SCOPED_NAMED_EVENT_TEXT("RenderSocialScreen_RenderThread()", FColor::Magenta);
 
@@ -240,6 +245,27 @@ void FDefaultSpectatorScreenController::RenderSpectatorScreen_RenderThread(FRHIC
 		SCOPED_DRAW_EVENT(RHICmdList, SpectatorScreen)
 		SpectatorScreenDelegate_RenderThread.Execute(RHICmdList, BackBuffer, SrcTexture, SpectatorScreenTexture_RenderThread, WindowSize);
 	}
+
+	// Apply the debug canvas layer.
+	IStereoLayers* StereoLayers = HMDDevice->GetStereoLayers();
+	if (StereoLayers)
+	{
+		const FIntRect DstRect(0, 0, BackBuffer->GetSizeX(), BackBuffer->GetSizeY());
+		for (int32 LayerID : DebugCanvasLayerIDs)
+		{
+			FTextureRHIRef LayerTexture = nullptr, HMDNull = nullptr;
+			StereoLayers->GetAllocatedTexture(LayerID, LayerTexture, HMDNull);
+			if (LayerTexture)
+			{
+				FTexture2DRHIRef LayerTexture2D = LayerTexture->GetTexture2D();
+				check(LayerTexture2D.IsValid());  // Debug canvas layer should be a 2d layer
+				const FIntRect LayerRect(0, 0, LayerTexture2D->GetSizeX(), LayerTexture2D->GetSizeY());
+				const FIntRect DstRectLetterboxed = Helpers::GetLetterboxedDestRect(LayerRect, DstRect);
+				HMDDevice->CopyTexture_RenderThread(RHICmdList, LayerTexture2D, LayerRect, BackBuffer, DstRectLetterboxed, false, false);
+			}
+		}
+		DebugCanvasLayerIDs.Empty();
+	}
 }
 
 
@@ -253,7 +279,7 @@ void FDefaultSpectatorScreenController::RenderSpectatorModeUndistorted(FRHIComma
 	const FIntRect SrcRect(0, 0, EyeTexture->GetSizeX(), EyeTexture->GetSizeY());
 	const FIntRect DstRect(0, 0, TargetTexture->GetSizeX(), TargetTexture->GetSizeY());
 
-	HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, SrcRect, TargetTexture, DstRect, false);
+	HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, SrcRect, TargetTexture, DstRect, false, true);
 }
 
 void FDefaultSpectatorScreenController::RenderSpectatorModeDistorted(FRHICommandListImmediate& RHICmdList, FTexture2DRHIRef TargetTexture, FTexture2DRHIRef EyeTexture, FTexture2DRHIRef OtherTexture, FVector2D WindowSize)
@@ -268,7 +294,7 @@ void FDefaultSpectatorScreenController::RenderSpectatorModeSingleEye(FRHICommand
 	const FIntRect SrcRect(0, 0, EyeTexture->GetSizeX() / 2, EyeTexture->GetSizeY());
 	const FIntRect DstRect(0, 0, TargetTexture->GetSizeX(), TargetTexture->GetSizeY());
 
-	HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, SrcRect, TargetTexture, DstRect, false);
+	HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, SrcRect, TargetTexture, DstRect, false, true);
 }
 
 void FDefaultSpectatorScreenController::RenderSpectatorModeSingleEyeLetterboxed(FRHICommandListImmediate& RHICmdList, FTexture2DRHIRef TargetTexture, FTexture2DRHIRef EyeTexture, FTexture2DRHIRef OtherTexture, FVector2D WindowSize)
@@ -277,7 +303,7 @@ void FDefaultSpectatorScreenController::RenderSpectatorModeSingleEyeLetterboxed(
 	const FIntRect DstRect(0, 0, TargetTexture->GetSizeX(), TargetTexture->GetSizeY());
 	const FIntRect DstRectLetterboxed = Helpers::GetLetterboxedDestRect(SrcRect, DstRect);
 
-	HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, SrcRect, TargetTexture, DstRectLetterboxed, true);
+	HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, SrcRect, TargetTexture, DstRectLetterboxed, true, true);
 }
 
 void FDefaultSpectatorScreenController::RenderSpectatorModeSingleEyeCroppedToFill(FRHICommandListImmediate& RHICmdList, FTexture2DRHIRef TargetTexture, FTexture2DRHIRef EyeTexture, FTexture2DRHIRef OtherTexture, FVector2D WindowSize)
@@ -288,7 +314,7 @@ void FDefaultSpectatorScreenController::RenderSpectatorModeSingleEyeCroppedToFil
 
 	const FIntRect SrcCroppedToFitRect = Helpers::GetEyeCroppedToFitRect(HMDDevice->GetEyeCenterPoint_RenderThread(EStereoscopicPass::eSSP_LEFT_EYE), SrcRect, WindowRect);
 
-	HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, SrcCroppedToFitRect, TargetTexture, DstRect, false);
+	HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, SrcCroppedToFitRect, TargetTexture, DstRect, false, true);
 }
 
 void FDefaultSpectatorScreenController::RenderSpectatorModeTexture(FRHICommandListImmediate& RHICmdList, FTexture2DRHIRef TargetTexture, FTexture2DRHIRef EyeTexture, FTexture2DRHIRef OtherTexture, FVector2D WindowSize)
@@ -302,7 +328,7 @@ void FDefaultSpectatorScreenController::RenderSpectatorModeTexture(FRHICommandLi
 	const FIntRect SrcRect(0, 0, SrcTexture->GetSizeX(), SrcTexture->GetSizeY());
 	const FIntRect DstRect(0, 0, TargetTexture->GetSizeX(), TargetTexture->GetSizeY());
 
-	HMDDevice->CopyTexture_RenderThread(RHICmdList, SrcTexture, SrcRect, TargetTexture, DstRect, false);
+	HMDDevice->CopyTexture_RenderThread(RHICmdList, SrcTexture, SrcRect, TargetTexture, DstRect, false, true);
 }
 
 void FDefaultSpectatorScreenController::RenderSpectatorModeMirrorAndTexture(FRHICommandListImmediate& RHICmdList, FTexture2DRHIRef TargetTexture, FTexture2DRHIRef EyeTexture, FTexture2DRHIRef OtherTexture, FVector2D WindowSize)
@@ -324,13 +350,13 @@ void FDefaultSpectatorScreenController::RenderSpectatorModeMirrorAndTexture(FRHI
 
 	if (SpectatorScreenModeTexturePlusEyeLayout_RenderThread.bDrawEyeFirst)
 	{
-		HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, CroppedEyeSrcRect, TargetTexture, EyeDstRect, bClearBlack);
-		HMDDevice->CopyTexture_RenderThread(RHICmdList, OtherTextureLocal, OtherSrcRect, TargetTexture, OtherDstRect, false);
+		HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, CroppedEyeSrcRect, TargetTexture, EyeDstRect, bClearBlack, true);
+		HMDDevice->CopyTexture_RenderThread(RHICmdList, OtherTextureLocal, OtherSrcRect, TargetTexture, OtherDstRect, false, true);
 	}
 	else
 	{
-		HMDDevice->CopyTexture_RenderThread(RHICmdList, OtherTextureLocal, OtherSrcRect, TargetTexture, OtherDstRect, bClearBlack);
-		HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, CroppedEyeSrcRect, TargetTexture, EyeDstRect, false);
+		HMDDevice->CopyTexture_RenderThread(RHICmdList, OtherTextureLocal, OtherSrcRect, TargetTexture, OtherDstRect, bClearBlack, true);
+		HMDDevice->CopyTexture_RenderThread(RHICmdList, EyeTexture, CroppedEyeSrcRect, TargetTexture, EyeDstRect, false, true);
 	}
 }
 

Engine/Source/Runtime/HeadMountedDisplay/Private/DefaultStereoLayers.cpp

@@ -313,3 +313,46 @@ void FDefaultStereoLayers::SetupViewFamily(FSceneViewFamily& InViewFamily)
 	HmdTransform = FTransform(HmdOrientation, HmdPosition);
 }
 
+void FDefaultStereoLayers::GetAllocatedTexture(uint32 LayerId, FTextureRHIRef &Texture, FTextureRHIRef &LeftTexture)
+{
+	Texture = LeftTexture = nullptr;
+	FLayerDesc* LayerFound = nullptr;
+
+	if (IsInRenderingThread())
+	{
+		for (int32 LayerIndex = 0; LayerIndex < RenderThreadLayers.Num(); LayerIndex++)
+		{
+			if (RenderThreadLayers[LayerIndex].GetLayerId() == LayerId)
+			{
+				LayerFound = &RenderThreadLayers[LayerIndex];
+			}
+		}
+	}
+
+	else
+	{
+		// Only supporting the use of this function on RenderingThread.
+		check(false);
+		return;
+	}
+
+	if (LayerFound && LayerFound->Texture)
+	{
+		switch (LayerFound->ShapeType)
+		{
+		case IStereoLayers::CubemapLayer:
+			Texture = LayerFound->Texture->GetTextureCube();
+			LeftTexture = LayerFound->LeftTexture ? LayerFound->LeftTexture->GetTextureCube() : nullptr;			
+			break;
+
+		case IStereoLayers::CylinderLayer:
+		case IStereoLayers::QuadLayer:
+			Texture = LayerFound->Texture->GetTexture2D();
+			LeftTexture = LayerFound->LeftTexture ? LayerFound->LeftTexture->GetTexture2D() : nullptr;
+			break;
+
+		default:
+			break;
+		}
+	}
+}

Engine/Source/Runtime/HeadMountedDisplay/Public/DefaultSpectatorScreenController.h

@@ -25,6 +25,7 @@ public:
 	virtual void SetSpectatorScreenTexture(UTexture* InTexture) override;
 	virtual UTexture* GetSpectatorScreenTexture() const override;
 	virtual void SetSpectatorScreenModeTexturePlusEyeLayout(const FSpectatorScreenModeTexturePlusEyeLayout& Layout) override;
+	virtual void QueueDebugCanvasLayerID(int32 LayerID) override;
 
 	FSpectatorScreenRenderDelegate* GetSpectatorScreenRenderDelegate_RenderThread();
 
@@ -32,7 +33,7 @@ public:
 	// Implementation methods called by HMD
 	virtual void BeginRenderViewFamily();
 	virtual void UpdateSpectatorScreenMode_RenderThread();
-	virtual void RenderSpectatorScreen_RenderThread(FRHICommandListImmediate& RHICmdList, FRHITexture2D* BackBuffer, FTexture2DRHIRef RenderTarget, FVector2D WindowSize) const;
+	virtual void RenderSpectatorScreen_RenderThread(FRHICommandListImmediate& RHICmdList, FRHITexture2D* BackBuffer, FTexture2DRHIRef RenderTarget, FVector2D WindowSize);
 
 protected:
 	friend struct FRHISetSpectatorScreenTexture;
@@ -63,6 +64,7 @@ protected:
 	FTexture2DRHIRef SpectatorScreenTexture_RenderThread;
 	FSpectatorScreenModeTexturePlusEyeLayout SpectatorScreenModeTexturePlusEyeLayout_RenderThread;
 	FSpectatorScreenRenderDelegate SpectatorScreenDelegate_RenderThread;
+	TArray<int32> DebugCanvasLayerIDs;
 
 	class HEADMOUNTEDDISPLAY_API Helpers
 	{

Engine/Source/Runtime/HeadMountedDisplay/Public/DefaultStereoLayers.h

@@ -26,6 +26,8 @@ public:
 	virtual void PostRenderView_RenderThread(FRHICommandListImmediate& RHICmdList, FSceneView& InView) override;
 	virtual bool IsActiveThisFrame(class FViewport* InViewport) const override;
 
+	void GetAllocatedTexture(uint32 LayerId, FTextureRHIRef &Texture, FTextureRHIRef &LeftTexture);
+
 protected:
 	
 	/**

Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayBase.h

@@ -49,7 +49,7 @@ public:
 	// Get the rectangle of the HMD rendertarget for the left eye which seems undistorted enough to be cropped and displayed on the spectator screen.
 	virtual FIntRect GetFullFlatEyeRect_RenderThread(FTexture2DRHIRef EyeTexture) const { return FIntRect(0, 0, 1, 1); }
 	// Helper to copy one render target into another for spectator screen display
-	virtual void CopyTexture_RenderThread(FRHICommandListImmediate& RHICmdList, FTexture2DRHIParamRef SrcTexture, FIntRect SrcRect, FTexture2DRHIParamRef DstTexture, FIntRect DstRect, bool bClearBlack) const {}
+	virtual void CopyTexture_RenderThread(FRHICommandListImmediate& RHICmdList, FTexture2DRHIParamRef SrcTexture, FIntRect SrcRect, FTexture2DRHIParamRef DstTexture, FIntRect DstRect, bool bClearBlack, bool bNoAlpha) const {}
 
 protected:
 	/**

Engine/Source/Runtime/HeadMountedDisplay/Public/IHeadMountedDisplay.h

@@ -115,6 +115,9 @@ public:
 	virtual void BeginRendering_GameThread() {}
 
 
+	// Are we outputting so a Spectator Screen now.
+	virtual bool IsSpectatorScreenActive() const { return false; }
+
 	/**
 	* Return a pointer to the SpectatorScreenController for the HMD if supported, else null.
 	* The controller is owned by the HMD, and will be destroyed when the HMD is destroyed.

Engine/Source/Runtime/HeadMountedDisplay/Public/ISpectatorScreenController.h

@@ -54,4 +54,6 @@ public:
 	* @param	bClearBlack: if true the render target will be drawn black before either rect is drawn.
 	*/
 	virtual void SetSpectatorScreenModeTexturePlusEyeLayout(const FSpectatorScreenModeTexturePlusEyeLayout& Layout) = 0;
+
+	virtual void QueueDebugCanvasLayerID(int32 LayerID) = 0;
 };

Engine/Source/Runtime/HeadMountedDisplay/Public/StereoLayerManager.h

@@ -14,6 +14,8 @@
 	The template argument should be a type for storing layer data. It should have a constructor matching the following:
 		LayerType(const FLayerDesc& InLayerDesc);
 	... and implement the following function overloads:
+		void SetLayerId(uint32 InId),
+		uint32 GetLayerId() const,
 		bool GetLayerDescMember(LayerType& Layer, FLayerDesc& OutLayerDesc),
 		void SetLayerDescMember(LayerType& Layer, const FLayerDesc& Desc), and
 		void MarkLayerTextureForUpdate(LayerType& Layer)
@@ -100,6 +102,7 @@ public:
 		uint32 LayerId = NextLayerId++;
 		check(LayerId > 0);
 		LayerType& NewLayer = StereoLayers.Emplace(LayerId, InLayerDesc);
+		NewLayer.SetLayerId(LayerId);
 		UpdateLayer(NewLayer, LayerId, InLayerDesc.Texture != nullptr);
 		bStereoLayersDirty = true;
 		return LayerId;
@@ -129,6 +132,7 @@ public:
 
 		LayerType& Layer = StereoLayers[LayerId];
 		SetLayerDescMember(Layer, InLayerDesc);
+		Layer.SetLayerId(LayerId);
 		UpdateLayer(Layer, LayerId, InLayerDesc.Texture != nullptr);
 		bStereoLayersDirty = true;
 	}