#rnx
#rb none
#ROBOMERGE-OWNER: ryan.durand
#ROBOMERGE-AUTHOR: ryan.durand
#ROBOMERGE-SOURCE: CL 10869210 via CL 10869511 via CL 10869900
#ROBOMERGE-BOT: (v613-10869866)
[CL 10870549 by ryan durand in Main branch]
#lockdown Nick.Penwarden
#rb none
==========================
MAJOR FEATURES + CHANGES
==========================
Change 3302759 on 2017/02/14 by John.Pollard
Remove invalid replay samples that can occur due to oversampling (sampling at higher rate than physics is being ticked)
Change 3306072 on 2017/02/16 by John.Barrett
Minor NetcodeUnitTest updates/fixes.
Improved assignment for some types, using FVMReflection - added assignment to enum by name.
Change 3322165 on 2017/02/24 by John.Barrett
Fixed issue where Steam would block unit tests. #JIRA UENET-537
Change 3323917 on 2017/02/27 by John.Barrett
Added better unit test log file management. #JIRA UENET-535
Unit test logs are now stored in subfolders in the game log folder, with one main folder for the current UE4 session, and one subfolder for every unit test run during that session.
Logs are purged occasionally, in line with how the main game logs are purged.
Change 3326307 on 2017/02/28 by John.Pollard
Remove unused cvar
Change 3327907 on 2017/03/01 by John.Barrett
Merged some old NetcodeUnitTest debug features:
- Added command for bit-based log dumping of data (for debugging bit-based netcode)
- Added exported function, for allowing access to console commands, in modules which don't import Engine
Change 3332710 on 2017/03/05 by John.Barrett
Updated unit test flags to expect a disconnect when expecting a server crash.
Change 3332715 on 2017/03/05 by John.Barrett
Wrapped all CreateChannelBunch pointers with a nullptr check (used to not be required).
Replaced old/duplicate unit test code (using CreateChannelBunch) for marking 'exploit failure', with the current centralized function calls.
#JIRA UENET-539
Change 3336258 on 2017/03/07 by John.Pollard
Add more info to history overflow logging
Change 3336259 on 2017/03/07 by John.Pollard
Don't pause replication for replay connections
Change 3341288 on 2017/03/10 by John.Barrett
Fixed invalid access to GLog during game shutdown. #JIRA UE-42394
Change 3341736 on 2017/03/10 by John.Pollard
Get reliability packet handler working again (but still disabled by default)
Change 3349298 on 2017/03/16 by John.Barrett
Fixed issues/potential-issues with the timing of global variable destruction, based on Graeme Thornton's fix. #JIRA UE-42394
Change 3349393 on 2017/03/16 by John.Pollard
Reliability handler cleanup
Change 3350029 on 2017/03/16 by John.Barrett
Updated PacketHandler to support an optional handshaking stage for HandlerComponents, where each handshake must execute sequentially starting at the component closest to the socket.
Change 3350030 on 2017/03/16 by John.Barrett
Updated Asymetric Encryption to support updated PacketHandler system, to perform a key handshake that is compatible with other HandlerComponent's, and fixed some serialization security issues by setting proper limits on values.
Change 3350032 on 2017/03/16 by John.Barrett
Minor corrections to variable types and constructor initializer lists, for the rest of the encryption code.
Change 3355536 on 2017/03/20 by John.Barrett
Added new PacketAudit debug feature, which uses inter-process-communication, to audit the integrity of incoming/outgoing packets at various stages of serialization, between a client and server.
This is critically useful for low level netcode development, particularly with the PacketHandler.
See PacketAudit.h for more information.
Change 3355570 on 2017/03/20 by John.Barrett
Added a small number packet auditing checks to key areas of low level netcode.
Change 3355584 on 2017/03/20 by John.Barrett
Updated ReliabilityHandlerComponent, to support packet reliability during the PacketHandler handshaking stage, and fixed ReliabilityHandlerComponent serializing based on a packets byte size, rather than bit size.
Change 3373389 on 2017/03/30 by John.Barrett
Added check for HandlerComponent's that require reliability.
Change 3373390 on 2017/03/30 by John.Barrett
Added TLS-like encryption handler component, which exchanges a secret key using asymmetric encryption (RSA), and implements symmetric encryption using that key (AES).
Based on RSA encryption component, and John Pollard's AES changes.
Enable in *Engine.ini, using:
[PacketHandlerComponents]
Components=RSAKeyAESEncryption
bEnableReliability=true
Change 3394518 on 2017/04/14 by Ryan.Gerleve
Fix memory leak in FRepChangelistState: added destructor that properly destroys the properties in the shadow state buffer.
Change 3432955 on 2017/05/10 by John.Barrett
Randomized the initial packet sequence numbers for UNetConnection's and for reliable packets - using the stateless handshake cookie.
Also added hooks to determine when PacketHandler level handshaking has completed, so that the initial connection packets can be delayed until this stage (required now, in order to exchange the sequence numbers first).
Change 3464528 on 2017/05/29 by John.Barrett
Fixed connection failure during unreliable network conditions, caused by a bad interaction between the stateless handshake and sequence initialization code.
Added a new 'ack' stage to the stateless handshake, to ensure the client/server sequence is properly synchronised.
Expanded the timeout/retry handling for the stateless handshake, so it is more robust, and fixed some bugs in this code.
Change 3464537 on 2017/05/29 by John.Barrett
Fixed an incompatibility between the asymmetric encryption, and the new PacketHandler handshake code.
Change 3464543 on 2017/05/29 by John.Barrett
Updated CryptoPP to 5.6.5.
Change 3467529 on 2017/05/31 by Jon.Nabozny
Add comments to FPacketSimulationSettings vars.
Change 3469584 on 2017/06/01 by Ryan.Gerleve
Log bunch ChSequence on reliable channel open along with other info.
Change 3471329 on 2017/06/02 by John.Barrett
Fixed PacketHandlerLog type not being exported.
Change 3471875 on 2017/06/02 by John.Barrett
Fixed code that was trying to send NetConnection packets from the server to the client, too early - and tidied up related checks.
#JIRA UE-45637
Change 3473030 on 2017/06/04 by John.Barrett
Fixed replay incomaptibility with recent sequence/handshake changes.
Change 3476313 on 2017/06/06 by Ryan.Gerleve
Add initial reliable sequence numbers to the log in UNetConnection::InitSequence.
Change 3478649 on 2017/06/07 by Ryan.Gerleve
Add additional logging for the case where a channel open command is received by a channel that's already opened locally.
Change 3482926 on 2017/06/09 by John.Barrett
First pass at Minimal Client refactor for NetcodeUnitTest - split basic/minimal client code out of ClientUnitTest, into its own class - while preserving existing ClientUnitTest interface.
This will allow multiple minimal clients per unit test, where only one could be used before (necessary for games that require a beacon + game connection).
Added reflection support for weak UObject properties.
Added MCP requirements flag for unit tests.
Process pipe reading tweaks, to avoid UI freezes.
Change 3485179 on 2017/06/12 by Ryan.Gerleve
Add backwards-compatibility macros for random initial packet sequence changes so that game plugins can check them.
Change 3489040 on 2017/06/14 by John.Barrett
Removed OnlineSubsystem dependencies.
[CL 3495525 by Ryan Gerleve in Main branch]
#lockdown Nick.Penwarden
#rb none
==========================
MAJOR FEATURES + CHANGES
==========================
Change 3209340 on 2016/11/23 by Ben.Marsh
Convert UE4 codebase to an "include what you use" model - where every header just includes the dependencies it needs, rather than every source file including large monolithic headers like Engine.h and UnrealEd.h.
Measured full rebuild times around 2x faster using XGE on Windows, and improvements of 25% or more for incremental builds and full rebuilds on most other platforms.
* Every header now includes everything it needs to compile.
* There's a CoreMinimal.h header that gets you a set of ubiquitous types from Core (eg. FString, FName, TArray, FVector, etc...). Most headers now include this first.
* There's a CoreTypes.h header that sets up primitive UE4 types and build macros (int32, PLATFORM_WIN64, etc...). All headers in Core include this first, as does CoreMinimal.h.
* Every .cpp file includes its matching .h file first.
* This helps validate that each header is including everything it needs to compile.
* No engine code includes a monolithic header such as Engine.h or UnrealEd.h any more.
* You will get a warning if you try to include one of these from the engine. They still exist for compatibility with game projects and do not produce warnings when included there.
* There have only been minor changes to our internal games down to accommodate these changes. The intent is for this to be as seamless as possible.
* No engine code explicitly includes a precompiled header any more.
* We still use PCHs, but they're force-included on the compiler command line by UnrealBuildTool instead. This lets us tune what they contain without breaking any existing include dependencies.
* PCHs are generated by a tool to get a statistical amount of coverage for the source files using it, and I've seeded the new shared PCHs to contain any header included by > 15% of source files.
Tool used to generate this transform is at Engine\Source\Programs\IncludeTool.
[CL 3209342 by Ben Marsh in Main branch]
==========================
MAJOR FEATURES + CHANGES
==========================
Change 2814968 on 2016/01/04 by John.Pollard
Remove warning per connection when same warning happens at net driver level
Change 2815271 on 2016/01/04 by John.Pollard
Fix header uploads in replays showing wrong size
#rb none
#tests Compiled, confirmed correct size
Change 2830613 on 2016/01/15 by John.Barrett
Added better ability for unit tests to test socket-level netcode
Full changes:
- Added flag for auto-reconnecting on connection close
- Added hook for socket level sends
- Added ability to block socket-level sends and pre-PacketHandler LowLevelSend's
- Added ability for net connections to force-enable packet handlers, even if presently off in the engine
#rb none
Change 2830617 on 2016/01/15 by John.Barrett@JC2Q-LT_UE4
Enabled NetcodeUnitTest in Linux, and other minor updates.
#rb none
Change 2830621 on 2016/01/15 by John.Barrett@JC2Q-LT_UE4
Added 'PacketLimitTest' and 'PacketLimitTest_Oodle' unit tests, for testing/verifying that MaxPacket limits are respected, through testing with Bunches and with raw sends passed through PacketHandlers (such as Oodle).
#rb none
Change 2832402 on 2016/01/18 by Bart.Bressler@bart.bressler_Dev_Networking
Always use the physical platform file when writing out steam_appid.txt, because it has to live next to executable that's being run (CotF builds will output it to the wrong location).
#rb josh.markiewicz
#tests checked that steam_appid.txt was written to the correct place with both CotF and cooked builds
Change 2833728 on 2016/01/19 by Joe.Conley@Joe.Conley_EGJWD5708_Dev-Networking-Minimal
Fix for UE-20017 - "Http server headers implemented as TMaps and are overridden". Just changing TMaps to TMultiMaps. Haven't modified the behavior of GetHeader(), which will still return the first header that matches.
Change 2844880 on 2016/01/26 by Joe.Conley
Fix for UE-20017 Http server headers implemented as TMaps and are overridden
Previously had submitted a fix that changes the request and response headers TMap into a MultiMap, but the consensus was that it would be better to leave them as TMaps, but allow appending onto the existing entry for a given request header with an "AppendToHeader" function, and to concatenate entries for the same header name in the response headers. In both cases, entries for the same header name are concatenated with ", ".
Mac and iOS already had the desired behavior for responses, just added the AppendToHeader function to their request classes.
HTML5 still has no implementation of response headers, filed as separate JIRA issue UE-26047.
#platformnotify josh.adams
Change 2859343 on 2016/02/08 by Bart.Bressler
Default MaxSearchResults in FOnlineSessionSearch objects to 1 instead of 0 - otherwise, the FOnlineAsyncTaskSteamFindServerBase tasks will complete right away without searching for sessions. Also print a warning in the task if this will happen.
#codereview josh.markiewicz
Change 2862352 on 2016/02/10 by Bart.Bressler
Fix demo.SkipTime cvar to work with negative values
#rb john.pollard
Change 2866624 on 2016/02/14 by John.Barrett
Fix plugin commandlet crash.
#rb none
Change 2866627 on 2016/02/14 by John.Barrett
Added NetDriver/PacketHandler ability to send/receive connectionless packets - i.e. packets not associated with a UNetConnection.
Removed 'bEnabled' from PacketHandler - PacketHandler is now a default always-on part of the Engine.
Added 'CountBits' to LowLevelSend, to support bit-level additions to packets from PacketHandler's, and refactored related code.
#codereview John.Pollard
Change 2866628 on 2016/02/14 by John.Barrett
Added an HMAC (Hash-based Message Authentication Code) generating function, to the SHA-1 code - for use with performing secure net connection handshakes.
#rb none
Change 2866629 on 2016/02/14 by John.Barrett
Added 'stateless handshake' to UE4, allowing a handshake to be performed without any open UNetConnection - making it more difficult for DoS attacks to consume memory resources.
#codereview John.Pollard
Change 2866630 on 2016/02/14 by John.Barrett
Updated Oodle to support new stateless handshake.
#rb none
Change 2866631 on 2016/02/14 by John.Barrett
Updated NetcodeUnitTest to work with new connectionless/stateless handshake changes. Compiles, but netcode presently needs fixing.
#rb none
Change 2868367 on 2016/02/16 by John.Barrett
Updated NetcodeUnitTest to work with stateless handshakes (now fixed connecting).
Also fixed further commandlet problems.
#rb none
Change 2878025 on 2016/02/23 by John.Barrett
Immediately stop processing zero-size packets, to prevent CPU utilization during DDoS.
Log IP of new connections, post-stateless-challenge (first time we know IP is verified).
#codereview john.pollard
Change 2883776 on 2016/02/26 by John.Barrett
Fixed beacons to work with stateless handshake.
Change 2886773 on 2016/02/29 by John.Barrett
Made PacketHandler bit overhead align upwards to the next byte - this wastes more bits than necessary, but is required as a temporary fix.
Change 2887801 on 2016/03/01 by John.Pollard
Use correct default values when initializing shadow state for replication.
Fixes issues with certain components and actors that have default values that differ from CDO.
[CL 2901490 by John Pollard in Main branch]
- Split [PacketHandlerComponents].Components ini setting, into an array, instead of one string.
- Give UNetConnection's a PacketHandler by default (for future handshake code).
- Refactored/cleaned-up a lot of PacketHandler code.
[CL 2711763 by John Barrett in Main branch]
