that is useful to be released early onto Main.
--------------------------
Bugfixes
--------------------------
Fix update default max bitrate to support 4k
Fix resolution changing and fix match viewport resolution browser side option.
Fix stream not playable when streamer disconnects but page not refreshed.
Fix SFU/data-only peers were able to request/gain quality controller status.
Fix delegate registration for backbuffer texture sources would sometimes be called when deleted and crash.
Fix uncessary bundling of pixel streaming servers when building non-game targets such as Editor/Server.
Fix PixelStreaming plugin now works with "-onethread" enabled and some methods have been made more thread safe.
--------------------------
Refactors
--------------------------
Refactor texture source API surface to enable texture sources that do not use Unreal Engine texture types.
Refactor to make FixedFPSPump be more usable by any arbitrary thing that wants updates at a fixed interval (not just video sources).
Refactor to remove player id from video sources to make them easier to create outside of a true peer connection existing.
Refactor to remove unecessary CVarPixelStreamingWaitForTextureCopy.
Refactor to expose functionality on module `CreateExternalVideoSource`. This is experimental and will likely be changed before 5.1.
--------------------------
Features
--------------------------
Added foundation for desktop streaming on Pixel Streaming.
Added simulcast support for VP8.
Added a compute shader for faster I420 conversion when using VPX, this is faster than CPU readback and conversion in our tests.
Added some PixelStreaming tests
1. A test that will start streaming, connect to a client, and check that a data channel message can be sent.
2. A mock signalling server test and functionality to test the C++ connection to the signalling server.
Added "stat pixelstreaming" and made PixelStreaming.HUDStats able to be toggled on/off.
Added custom UE implementation webrtc::AdaptedVideoTrackSource that can add/remove remove video sinks at runtime.
Added ability on module to start/stop streaming as needed.
Added peers can now swap roles between P2P and simulcast mid-stream.
To test this new feature:
1. Simply toggle the PreferSFU toggle in the settings panel.
2. Click the new restart stream button in the settings panel.
#JIRA UCS-2781,UE-140913,UE-141101,UE-141104,UE-141105,UE-141106,UE-141862,UE-147157
#rb Luke.Bermingham
#fyi Nick.Pace, Mathew.Cotton, Mattias.Jansson
#preflight 624a7c27637925b5d3f7840c
#preflight 624b71109f404234149a58a2
[CL 19616813 by Aidan Possemiers in ue5-main branch]
- Adding IVulkanDynamicRHI interface for plugins that want to touch Vulkan resources/handles directly.
- Modifying plugins that were using VulkanRHIPrivate.h to use the interface instead.
- Removing plugin references to UE Modules and ThirdParty libraries that were only needed because they included the private RHI headers.
- Removing VulkanRHI/Private from plugin include paths.
- Deprecating VulkanRHIBridge and moved its functionality into the new interface.
#jira none
#rb jeannoe.morissette, mihnea.balta, robert.srinivasiah
#preflight 62281ec531133a23da642007
#robomerge FNNC
[CL 19320995 by christopher waters in ue5-main branch]
- Adding ID3D12DynamicRHI and ID3D11DynamicRHI interfaces for plugins that want to touch DirectX resources/devices directly.
- Adding specific headers for including DirectX headers: D3D12ThirdParty.h and D3D11ThirdParty.h
- Modifying plugins that were using D3D11RHIPrivate.h and D3D12RHIPrivate.h to use the interfaces instead.
- Removing plugin references to UE Modules and ThirdParty libraries that were only needed because they included the private RHI headers.
- Removing D3D11RHI/Private and D3D12RHI/Private from plugin include paths.
#jira none
#rb jeannoe.morissette, kenzo.terelst, mihnea.balta, eric.mcdaniel
#preflight 6222395bb066ef60bbad004a
[CL 19267223 by christopher waters in ue5-main branch]
Refactor player sessions
Fix PixelStreamingAudioComponent had delayed audio.
Fix audio bitrate from Unreal Engine to browser was set to a low default causing compressed audio quality in stream. The new default is now Opus maximum value of 510kb/s.
Fix the Selective Forwarding Unit (SFU) not being added to the Pixel Streaming samples folder for a packaged project
Fix default max bitrate not being high enough to support 4k.
Fix reconnections not autoplaying in the browser.
Add datachannel support to SFU.
Add WebRTC c++ client behaviour to pixel streaming system. This is used for developing unit tests.
Add a unit test that will start streaming, connect a client and check that a data channel message can be sent.
Add the ability for a user to start/stop streaming as needed through the use of PixelStreaming.StartStreaming and PixelStreaming.StopStreaming
Add the ability for the stream resolution to be changed at runtime
Notes: AVEncoder no longer stores a width and height; These properties have been moved to the FVideoEncoderInput. This input is now the source of truth and both AMF and NVENC will adapt their resolution to the input resolution.
#rb luke.bermingham
#fyi mattias.jansson, nick.pace, matthew.cotton, aidan.possemiers, sandor.hadas
#preflight 6204c7aba65a8a28464df03c
#jira none
[CL 18948482 by William Belcher in ue5-main branch]
- Vulkan AMF support is currently unsupported and has been disabled.
- Removed fatal error when trying to use AMF with Vulkan.
#JIRA UE-133434
#rb Luke.Bermingham
[FYI] Alejandro.Arango, Brandon Schaefer, Mattias.Jansson
#preflight 61f882ed80608c7029a0add1
#ROBOMERGE-AUTHOR: aidan.possemiers
#ROBOMERGE-SOURCE: CL 18803094 in //UE5/Release-5.0/... via CL 18803106 via CL 18821573
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)
[CL 18821656 by aidan possemiers in ue5-main branch]
- Adds /?preferSFU option for browser peers to indicate they wish to use the SFU
- Adds an SFU server to Samples/PixelStreaming/WebServers/SFU
- Renames many classes in the PixelStreaming plugin to remove the "PixelStreaming" prefix from class names in private classes in favour of the namespace UE::PixelStreaming; public classes retain the prefix PixelStreaming and do not use the namespace.
- Renamed a number of classes to use the phrase "PixelStreaming" instead of "PixelStreamer", specifically, PixelStreamerInputComponent to PixelStreamingInputComponent
- All blueprint exposed static functions have been moved into a blueprint function library in PixelStreamingBlueprints.
- Fixed a memory leak caused by buffer allocation in the NVENC encoder that was never emptied.
- Moved to an explicit single threaded model for the NVENC video encoder - meaning only one encode job at a time per instance of the encoder.
- Moved to more explicit failure message when a user runs out of available NVENC sessions - this is common a consumer NVIDIA GPUs that only support 3 encoding sessions now that we are supporting simulcast.
- Introduced default simulcast settings of two streams: a full res stream and a half res stream. This can be overridden with `-SimulcastParameters=1.0,5000000,20000000,2.0,1000000,5000000` where each stream's parameters are specified in triples of scaling factor, min bitrate, max bitrate. Beware of running out of NVENC sessions if you specify too many stream sizes.
- Moved to more safety checking during shutdown of datachannel observer as this class was causing some intermittent failures to do a clean shutdown.
- Exposed WebRTC stats to Pixel Streaming. These can be viewed on screen using `-PixelStreamingHUDStats`.
#JIRA UE-139687 UCS-2781
#rb Luke.Bermingham
[FYI] Mattias.Jansson, Michael.Wallace
#preflight 61e8a6dd817064a62d3ac148
#ROBOMERGE-AUTHOR: aidan.possemiers
#ROBOMERGE-SOURCE: CL 18670627 in //UE5/Release-5.0/... via CL 18670647 via CL 18670649
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v901-18665521)
[CL 18670653 by aidan possemiers in ue5-main branch]
This resolves bitrate issues in WebRTC caused by inconsistent frame rates.
These changes are intended for 5.0 and licencees who are using source builds of the engine to get the updates before 5.0 is released.
#JIRA UCS-2077
#rb Aidan.Possemiers
[FYI] Luke.Bermingham, Nick.Pace, Matthew.Cotton, Marco.Anastasi
#lockdown Mitchell.Wilson
#ROBOMERGE-AUTHOR: aidan.possemiers
#ROBOMERGE-SOURCE: CL 18267227 in //UE4/Main/... via CL 18267244 via CL 18267255
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v895-18170469)
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 18267257 by aidan possemiers in ue5-release-engine-test branch]
