*************
Features
*************
- Added `stat pixelstreaminggraphs` that show realtime graphs of Pixel Streaming stats.
- Added -PixelStreamingInputController=Host (or Any) to optionally restrict stream inputs to a single host peer. The default is -PixelStreamingInputController=Any
- Add PixelStreamingServers utility to launch PixelStreaming servers from C++. (Added accompanying test too).
- Added tests for FPixelStreamingSourceFrame, FPixelStreamingFrameMetadata, FPixelStreamingFrameAdapterProcess and FPixelStreamingFrameAdapter
- Added the ability to override the default InputDevice.
- Added the ability to have multiple streamers.
*************
Bugfixes
*************
- Fix so that NVENC now only needs to scale frames as needed, there were some extraneous scaling happening before.
- Fix on screen hud stats on startup that were not dismissable.
- Fix PixelStreaming module state cleanup that was causing packaging crashes.
- Fix bug where SFU peers were unable to receive data channels messages from the streamer.
- Fix failing "connect" test where quality & input controller messages were being sent too early (before datachannel was established) for P2P peers.
- Fix PixelStreamingAudioComponent finding trying to find audio sink on a data-only peer and crashing.
- Fix PIXELSTREAMING_API exports on some files.
- Fix PixelStreaming.build.cs that was too harsh about excluding any file with 'node' in the name.
- Fix PixelStreaming.build.cs to exclude packaging the platform_scripts/cmd/coturn dir.
- Fix Vulkan fast texture copy codepath to not crash in ssome edgecases.
*************
Refactors
*************
- Frames now flow through a much more rigid set of operations to get streamed.
- Video Sources have a Video Adapter and are fed by a Video Input.
- Video Adapters are fed by the input and contains a number of `Processes`.
- Processes take the input frame and adapt it to a format that the encoder can use.
- Frame Buffers now contains a reference to the adapter, frame data is grabbed from here as needed by the encoder.
- Adapters and their processes are also public with the intention that users can implement their own adapt step for other encoders.
- Streamer is also now modular so we can have multiple streamers with their own inputs that stream to their own location.
- Extracted out the copy texture into a rendering util and tweaked how it handles transitions.
- Modify PixelStreamingModule to use new functions for creating a streamer.
- Added optional codepaths to use WebRTC M96 or M84.
- Refactor plugin to enable using our API for the new Pixel Streaming Player plugin.
- Created a new PixelStreamingPeerConnection class to wrap the webrtc peer connection with specific pixelstreaming behaviour.
- Made the signalling server connection public and renamed its observer methods so its obvious when they're implemented.
- Some small changes to frame buffers so make handling of cached data slightly safer.
- Made use of lambdas more explicit where possible.
- Remove unnecessary warning message from PixelStreamingAudioComponent.
#rb Luke.Bermingham
#fyi Mattias.Jansson, Nick.Pace, Mathew.Cotton
#jira UE-152859
#preflight 6289bd911422fb01af9601b7
[CL 20315933 by Aidan Possemiers in ue5-main branch]
that is useful to be released early onto Main.
--------------------------
Bugfixes
--------------------------
Fix update default max bitrate to support 4k
Fix resolution changing and fix match viewport resolution browser side option.
Fix stream not playable when streamer disconnects but page not refreshed.
Fix SFU/data-only peers were able to request/gain quality controller status.
Fix delegate registration for backbuffer texture sources would sometimes be called when deleted and crash.
Fix uncessary bundling of pixel streaming servers when building non-game targets such as Editor/Server.
Fix PixelStreaming plugin now works with "-onethread" enabled and some methods have been made more thread safe.
--------------------------
Refactors
--------------------------
Refactor texture source API surface to enable texture sources that do not use Unreal Engine texture types.
Refactor to make FixedFPSPump be more usable by any arbitrary thing that wants updates at a fixed interval (not just video sources).
Refactor to remove player id from video sources to make them easier to create outside of a true peer connection existing.
Refactor to remove unecessary CVarPixelStreamingWaitForTextureCopy.
Refactor to expose functionality on module `CreateExternalVideoSource`. This is experimental and will likely be changed before 5.1.
--------------------------
Features
--------------------------
Added foundation for desktop streaming on Pixel Streaming.
Added simulcast support for VP8.
Added a compute shader for faster I420 conversion when using VPX, this is faster than CPU readback and conversion in our tests.
Added some PixelStreaming tests
1. A test that will start streaming, connect to a client, and check that a data channel message can be sent.
2. A mock signalling server test and functionality to test the C++ connection to the signalling server.
Added "stat pixelstreaming" and made PixelStreaming.HUDStats able to be toggled on/off.
Added custom UE implementation webrtc::AdaptedVideoTrackSource that can add/remove remove video sinks at runtime.
Added ability on module to start/stop streaming as needed.
Added peers can now swap roles between P2P and simulcast mid-stream.
To test this new feature:
1. Simply toggle the PreferSFU toggle in the settings panel.
2. Click the new restart stream button in the settings panel.
#JIRA UCS-2781,UE-140913,UE-141101,UE-141104,UE-141105,UE-141106,UE-141862,UE-147157
#rb Luke.Bermingham
#fyi Nick.Pace, Mathew.Cotton, Mattias.Jansson
#preflight 624a7c27637925b5d3f7840c
#preflight 624b71109f404234149a58a2
[CL 19616813 by Aidan Possemiers in ue5-main branch]
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485
[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
Brings in multiple improvements such as parameterisation of many encoder settings, AMF encoder, better pixel streaming stats and some bugfixes
#JIRA
#rb self
[FYI] marco.anastasi, luke.bermingham, matthew.cotton, aidan.possemiers
#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: nick.pace
#ROBOMERGE-SOURCE: CL 16433855 in //UE4/Release-4.27/... via CL 16433858
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Staging) (v814-16434418)
#ROBOMERGE-CONFLICT from-shelf
[CL 16435808 by marc audy in ue5-release-engine-staging branch]
