124 Commits

Author SHA1 Message Date
Graeme Thornton
1426faf9b3 Created a new interface to RSA functionality that is independent of key size (unlike the old TBigInt version). Internally, RSA functions are implemented by OpenSSL on Windows/Mac/Linux an falling back to the TBigInt implementation for other platforms
- Future plan would be not to use OpenSSL directly, but to go via PlatformCrypto or something like it which could provide platform specific RSA functionality if it exists
- Also contains some compensation code for platforms that are still stuck on an older version of OpenSSL (everything except windows)
- On platforms that use OpenSSL < 1.1.1, register a locking callback so we can provide mutexes and stop it crashing horribly when the same RSA key is used concurrently on different threads

Changed default key size in the CryptoKeys plugin to 4096 bits.

#jira UE-71377
#rb ian.fox, robert.manuszewski, ryan.gerleve

[CL 5447433 by Graeme Thornton in 4.22 branch]
2019-03-19 07:30:08 -04:00
Graeme Thornton
b5c9ae46ac Disable SHA1 hashes for pak signing. Still experimental, and not indended to be enabled by default
#rb none
#jira UE-71377

[CL 5383199 by Graeme Thornton in 4.22 branch]
2019-03-13 11:24:00 -04:00
Graeme Thornton
0470468f87 Improvements to RSA key generation and pak signing
- CryptoKeys plugin can now be told how many bits to use for keys. Defaults to 2048 as a happy medium between security and performance
- After creating the OpenSSL RSA key, use the exp/mod values directly rather than recomputing from primes
- Added RSA.h with some TBigInt based helpers for RSA keys
- Added FPakSignatureFile to wrap the contents of the pak .sig file, including determining that it is of a new format that supports versioning. Wraps the encryption and decryption of he master table hash
- Changed pak master signature hash from crc to SHA1
- Modified access to executable embedded public key data so that it is just an array of bytes. Will allow custom key sizes in the future.

#rb joe.barrett, stefan.boberg
#jira UE-71280, UE-71377

[CL 5381335 by Graeme Thornton in 4.22 branch]
2019-03-13 09:47:41 -04:00
peter knepley
ae04c536e6 Remove PakFile layer from the platform file chain upon shutdown (FORT-145923)
- Prevents CRC from crashing when writing the config files on exit.



#rnx


#rb Josh.Adams, Brandon.Schaefer


#ROBOMERGE-AUTHOR: Arciel.Rekman
#ROBOMERGE-SOURCE: CL 4846937 via CL 4846952 via CL 4848372 via CL 4850934

[CL 4851162 by peter knepley in Main branch]
2019-01-30 18:56:57 -05:00
Josh Adams
0cd6ef2a9c - Fixed conflict from a merge to main
#rb trivial
#jira none

[CL 4846991 by Josh Adams in Main branch]
2019-01-30 15:27:53 -05:00
josh adams
87bcd24cc3 [Gil.Gribb and maybe others did most of the work, I just did the IOS parts, but I am checking it in, since IOS was the last stop on the code train]
- Added memory mapped IO concept to the engine
- Implemented and tested on Windows and IOS
- Not in use by anything yet, however, as the use cases are narrow and special (bulk data that does not need massaging after load - so it can be swapped out and back in silently by OS)
[FYI] gil.gribb,chris.babcock


#ROBOMERGE-SOURCE: CL 4842051 via CL 4845281

[CL 4845299 by josh adams in Main branch]
2019-01-30 13:45:30 -05:00
graeme thornton
8c534fe278 When a shader pak reader is first created for a thread, return the FThreadCheckingArchiveProxy wrapped version rather than the raw archive pointer.
#ROBOMERGE-SOURCE: CL 4829903 via CL 4829916 via CL 4830765

[CL 4830771 by graeme thornton in Main branch]
2019-01-29 08:11:08 -05:00
graeme thornton
c0bae37e42 Pak file handles no longer store a pointer to a shared reader archive. Those readers are thread specific and means handles can't be read from different threads without causing problems. By storing a function that looks up the pak reader dynamically on each file operation, the requests will work properly from any thread.
#rb gil.gribb


#ROBOMERGE-SOURCE: CL 4829513 via CL 4829541 via CL 4830764

[CL 4830770 by graeme thornton in Main branch]
2019-01-29 08:11:07 -05:00
graeme thornton
f3c891253b Removed ensure from signedarchivereader pak signature failure handler to make it consistent with the pakprecacher version. Games can override the signature check failure delegate to choose what to do when a signature fails
#ROBOMERGE-SOURCE: CL 4822286 via CL 4822301 via CL 4822810

[CL 4822827 by graeme thornton in Main branch]
2019-01-28 10:35:42 -05:00
Robert Manuszewski
cfa01c09c7 Merging //UE4/Dev-Main to Dev-Core (//UE4/Dev-Core)
#rb none

[CL 4731457 by Robert Manuszewski in Dev-Core branch]
2019-01-16 04:23:12 -05:00
Francis Hurteau
57c677da93 Copying //UE4/Dev-Enterprise@4705006 to Dev-Main (//UE4/Dev-Main)
#rb none
#lockdown Nick.Penwarden

[CL 4705151 by Francis Hurteau in Main branch]
2019-01-10 17:26:53 -05:00
Marc Audy
ab108c215e Copying to Dev-Main @ CL# 4688301
#rb
#rnx

[CL 4688816 by Marc Audy in Main branch]
2019-01-08 11:38:48 -05:00
Robert Manuszewski
2752c82adc Merging //UE4/Dev-Main @ 4664414 to Dev-Core (//UE4/Dev-Core)
#rb none

[CL 4675693 by Robert Manuszewski in Dev-Core branch]
2019-01-02 00:55:51 -05:00
Ben Marsh
7598af0532 Update copyright notices to 2019.
#rb none
#lockdown Nick.Penwarden

[CL 4662404 by Ben Marsh in Main branch]
2018-12-14 13:41:00 -05:00
Robert Manuszewski
1685ed0b2e Merging //UE4/Dev-Main @ 4620382 to Dev-Core (//UE4/Dev-Core)
#rb none

[CL 4631078 by Robert Manuszewski in Dev-Core branch]
2018-12-04 23:30:42 -05:00
Graeme Thornton
1b1e333282 Fix an error in FPakEntry::GetSerializedSize() which was incorrect for versions before the named compression mode stuff
#rb none
#fyi josh.adams, steve.robb

[CL 4589978 by Graeme Thornton in Dev-Core branch]
2018-11-23 05:52:01 -05:00
Ben Marsh
ea340d3fbf Copying //UE4/Fortnite-Staging to Dev-Main (//UE4/Dev-Main)
#rb none
#lockdown Nick.Penwarden

[CL 4567513 by Ben Marsh in Main branch]
2018-11-14 19:05:13 -05:00
Robert Manuszewski
d299e8f501 Merging //UE4/Dev-Main @ 4506166 to Dev-Core (//UE4/Dev-Core)
#rb none

[CL 4506617 by Robert Manuszewski in Dev-Core branch]
2018-10-25 08:06:47 -04:00
marcus wassmer
af89a63671 Duplicating 4461277
Workaround for UE-58013, though not a full fix.  Requires setting bAllowNonUFSIniWhenCooked in the Target.cs and using a cooked build with pak files.
#rb bob.tellez
#jira UE-58013

#ROBOMERGE-SOURCE: CL 4487659 in //UE4/Release-4.21/...
#ROBOMERGE-BOT: RELEASE (Release-4.21 -> Release-Staging-4.21)

[CL 4487660 by marcus wassmer in Staging-4.21 branch]
2018-10-18 15:19:05 -04:00
Josh Adams
4f63a76b48 - Changed compression methods to be an FName instead of hardcoded enum
- Added support for compression plugins
- Removed the Custom compression concept, now using plugins properly
- Modified UnrealPak to use FNames, and allow for multiple compression methods (fallbacks on error or unavailability, etc)
- Added project settings for compression method selection for UnrealPak, and additional settings to be passed to UnrealPak (for instance, to control compression size/speed, etc)
- Deprecated a bunch of old function calls
- Improved pak file "old format" reading ability
- Brought over some changes from Fortnite for pak file encryption and memory savings
- Implemented a parallel compression pull request (#4129) to speed up pak file compression
#jira UE-51294
#rb ben.marsh

[CL 4480944 by Josh Adams in Dev-Core branch]
2018-10-17 14:18:10 -04:00
Robert Manuszewski
02ba9f72df Merging //UE4/Dev-Main @ 4397329 to Dev-Core (//UE4/Dev-Core)
#rb none

[CL 4397828 by Robert Manuszewski in Dev-Core branch]
2018-09-26 08:22:54 -04:00
Marc Audy
af90b7bcd4 Copying //UE4/Fortnite-Staging to Dev-Main (//UE4/Dev-Main) @ 4395008
#rb
#rnx
#lockdown Nick.Penwarden

[CL 4395058 by Marc Audy in Main branch]
2018-09-25 10:11:35 -04:00
Robert Manuszewski
ec82e184bd Merging //UE4/Dev-Main @ 4356719 to Dev-Core (//UE4/Dev-Core)
#rb none

[CL 4357040 by Robert Manuszewski in Dev-Core branch]
2018-09-11 02:45:29 -04:00
Jack Porter
079be7f538 Merging //UE4/Dev-Main to Dev-Mobile (//UE4/Dev-Mobile)
#rb None
#jira 0

[CL 4293080 by Jack Porter in Dev-Mobile branch]
2018-08-16 13:53:43 -04:00
Ben Marsh
06319fa85a Add a bCompileAgainstApplicationCore setting to the target rules, which allows compiling out references to the ApplicationCore module (which should only be necessary for applications with a GUI). Removed ApplicationCore from several engine tools and utilities.
#rb none

[CL 4224708 by Ben Marsh in Dev-Core branch]
2018-07-20 09:08:07 -04:00