Commit Graph

116 Commits

Author SHA1 Message Date
hongyi yu
4e39844ff7 Fixed the issue where unrealpak failed to read from old pak files with no content.
#rb Graeme.Thornton

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: hongyi.yu
#ROBOMERGE-SOURCE: CL 5484250 via CL 5487362 via CL 5487860
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5507539 by hongyi yu in Dev-Networking branch]
2019-03-22 17:12:24 -04:00
graeme thornton
31bb0a2496 Edigrated rollup of all engine RSA changes that were done for 4.22. Pak signature checking is now much more competent.
#rb none
[FYI] joe.barrett
#rnx

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5469067 via CL 5469119 via CL 5470585 via CL 5470597
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5505358 by graeme thornton in Dev-Networking branch]
2019-03-22 16:59:58 -04:00
graeme thornton
21adb81f48 Created a new interface to RSA functionality that is independent of key size (unlike the old TBigInt version). Internally, RSA functions are implemented by OpenSSL on Windows/Mac/Linux an falling back to the TBigInt implementation for other platforms
- Future plan would be not to use OpenSSL directly, but to go via PlatformCrypto or something like it which could provide platform specific RSA functionality if it exists
- Also contains some compensation code for platforms that are still stuck on an older version of OpenSSL (everything except windows)
- On platforms that use OpenSSL < 1.1.1, register a locking callback so we can provide mutexes and stop it crashing horribly when the same RSA key is used concurrently on different threads

Changed default key size in the CryptoKeys plugin to 4096 bits.

#jira UE-71377
#rb ian.fox, robert.manuszewski, ryan.gerleve

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5447433 in //UE4/Release-4.22/... via CL 5447485
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5447724 by graeme thornton in Dev-Networking branch]
2019-03-19 09:01:05 -04:00
graeme thornton
5e5a675bd3 Disable SHA1 hashes for pak signing. Still experimental, and not indended to be enabled by default
#rb none
#jira UE-71377

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5383199 in //UE4/Release-4.22/... via CL 5383328
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5415581 by graeme thornton in Dev-Networking branch]
2019-03-15 12:02:39 -04:00
graeme thornton
a374031f9f Improvements to RSA key generation and pak signing
- CryptoKeys plugin can now be told how many bits to use for keys. Defaults to 2048 as a happy medium between security and performance
- After creating the OpenSSL RSA key, use the exp/mod values directly rather than recomputing from primes
- Added RSA.h with some TBigInt based helpers for RSA keys
- Added FPakSignatureFile to wrap the contents of the pak .sig file, including determining that it is of a new format that supports versioning. Wraps the encryption and decryption of he master table hash
- Changed pak master signature hash from crc to SHA1
- Modified access to executable embedded public key data so that it is just an array of bytes. Will allow custom key sizes in the future.

#rb joe.barrett, stefan.boberg
#jira UE-71280, UE-71377

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5381335 in //UE4/Release-4.22/... via CL 5383057
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5415560 by graeme thornton in Dev-Networking branch]
2019-03-15 12:02:13 -04:00
allan bentham
1c4c9c8fe4 Fix comments somehow lost during merge
#jira UE-70963
#rb none

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: allan.bentham
#ROBOMERGE-SOURCE: CL 5294300 in //UE4/Main/...
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5303312 by allan bentham in Dev-Networking branch]
2019-03-05 15:37:24 -05:00
allan bentham
1d07100d43 Optimize deserialization of compressed pak files.
Reuse previously decompressed buffer where possible to avoid repeatedly deserialising and decompressing the same block when serializing data < compressed block size.
#rb Graeme.Thornton
[FYI] Graeme.Thornton

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: allan.bentham
#ROBOMERGE-SOURCE: CL 5237090 via CL 5240033 via CL 5240035
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5255990 by allan bentham in Dev-Networking branch]
2019-02-28 18:26:25 -05:00
peter sauerbrei
fe05315464 addition of better hash algorithm for removal of file names from hash
#rb josh.adams

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: peter.sauerbrei
#ROBOMERGE-SOURCE: CL 5075321 via CL 5075990 via CL 5081503 via CL 5081850
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5137391 by peter sauerbrei in Dev-Networking branch]
2019-02-22 04:27:31 -05:00
josh adams
faf3c16242 - Reenabled encryption on bulk files with memory mapping - and then disallow mapping at runtime
#rb gil.bribb
[FYI] gil.gribb

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 4998359 via CL 4998360 via CL 4999252 via CL 5001374
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5084887 by josh adams in Dev-Networking branch]
2019-02-19 19:37:48 -05:00
josh adams
c510e3cc7b - Memory Mapped IO enabled on IOS for Audio (via ini and platform property)
- -nommio on the commandline will disble it for testing
#rb gil.gribb

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 4963115 via CL 4963118 via CL 4963812 via CL 4969283
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5074095 by josh adams in Dev-Networking branch]
2019-02-19 17:29:04 -05:00
justin marcus
a72aa3c306 [CODEREVIEW] justin.marcus,jeff.williams
Merge conflict.
justin.marcus, please merge this change by hand.
More info at https://robomerge.epicgames.net#FORTNITE

/src/robomerge_ue4_main_01/Engine/Source/Runtime/Core/Private/GenericPlatform/GenericPlatformMisc.cpp - merging //Fortnite/Dev-EngineMerge/Engine/Source/Runtime/Core/Private/GenericPlatform/GenericPlatformMisc.cpp#3
/src/robomerge_ue4_main_01/Engine/Source/Runtime/Core/Private/GenericPlatform/GenericWidePlatformString.cpp - merging //Fortnite/Dev-EngineMerge/Engine/Source/Runtime/Core/Private/GenericPlatform/GenericWidePlatformString.cpp#4
/src/robomerge_ue4_main_01/Engine/Source/Runtime/Core/Public/GenericPlatform/GenericPlatformMisc.h - merging //Fortnite/Dev-EngineMerge/Engine/Source/Runtime/Core/Public/GenericPlatform/GenericPlatformMisc.h#3


--------------------------------------
Merging //Fortnite/Dev-Kairos to Main (//Fortnite/Main)

Get Install Bundle Manager and dependcies building in main.




#rb none

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: justin.marcus
#ROBOMERGE-SOURCE: CL 4949345 via CL 4952070 via CL 4961624
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 5073469 by justin marcus in Dev-Networking branch]
2019-02-19 17:16:21 -05:00
peter knepley
a78de2c339 Remove PakFile layer from the platform file chain upon shutdown (FORT-145923)
- Prevents CRC from crashing when writing the config files on exit.



#rnx


#rb Josh.Adams, Brandon.Schaefer

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: Arciel.Rekman
#ROBOMERGE-SOURCE: CL 4846937 via CL 4846952 via CL 4848372 via CL 4850934 via CL 4851162
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4851726 by peter knepley in Dev-Networking branch]
2019-01-30 18:58:55 -05:00
josh adams
acdf15c548 - Fixed conflict from a merge to main
#rb trivial
#jira none

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 4846991 in //UE4/Main/...
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4847000 by josh adams in Dev-Networking branch]
2019-01-30 15:28:22 -05:00
josh adams
5088bccb91 [Gil.Gribb and maybe others did most of the work, I just did the IOS parts, but I am checking it in, since IOS was the last stop on the code train]
- Added memory mapped IO concept to the engine
- Implemented and tested on Windows and IOS
- Not in use by anything yet, however, as the use cases are narrow and special (bulk data that does not need massaging after load - so it can be swapped out and back in silently by OS)
[FYI] gil.gribb,chris.babcock

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 4842051 via CL 4845281 via CL 4845299
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4845422 by josh adams in Dev-Networking branch]
2019-01-30 13:46:29 -05:00
graeme thornton
9830f4ad2d When a shader pak reader is first created for a thread, return the FThreadCheckingArchiveProxy wrapped version rather than the raw archive pointer.
#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 4829903 via CL 4829916 via CL 4830765 via CL 4830771
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4830777 by graeme thornton in Dev-Networking branch]
2019-01-29 08:11:14 -05:00
graeme thornton
e789d1fda7 Pak file handles no longer store a pointer to a shared reader archive. Those readers are thread specific and means handles can't be read from different threads without causing problems. By storing a function that looks up the pak reader dynamically on each file operation, the requests will work properly from any thread.
#rb gil.gribb

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 4829513 via CL 4829541 via CL 4830764 via CL 4830770
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4830776 by graeme thornton in Dev-Networking branch]
2019-01-29 08:11:13 -05:00
graeme thornton
993960ddc8 Removed ensure from signedarchivereader pak signature failure handler to make it consistent with the pakprecacher version. Games can override the signature check failure delegate to choose what to do when a signature fails
#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 4822286 via CL 4822301 via CL 4822810 via CL 4822827
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4822834 by graeme thornton in Dev-Networking branch]
2019-01-28 10:35:49 -05:00
robert manuszewski
be36beb7e4 Copying //UE4/Dev-Core [at] 4771952 to Dev-Main (//UE4/Dev-Main)
#rb none
#lockdown Nick.Penwarden

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 4772220 in //UE4/Main/...
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4782699 by robert manuszewski in Dev-Networking branch]
2019-01-23 09:49:18 -05:00
ryan gerleve
b41214417e Resolving conflict from Dev-Enterprise.
/src/ROBOMERGE_ENGINE_Dev_Networking/Engine/Source/Runtime/Sockets/Private/BSDSockets/SocketsBSD.cpp - merging //UE4/Main/Engine/Source/Runtime/Sockets/Private/BSDSockets/SocketsBSD.cpp#20


--------------------------------------
Copying //UE4/Dev-Enterprise[at]4705006 to Dev-Main (//UE4/Dev-Main)

#rb none
#lockdown Nick.Penwarden

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 4705151 in //UE4/Main/...
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4712032 by ryan gerleve in Dev-Networking branch]
2019-01-11 14:26:15 -05:00
James Hopkin
235137b0cc Merging //UE4/Dev-Main to Dev-Networking (//UE4/Dev-Networking)
Copying to Dev-Main @ CL# 4688301

#rb none

[CL 4707991 by James Hopkin in Dev-Networking branch]
2019-01-11 02:57:03 -05:00
ben marsh
2b46ba7b94 Update copyright notices to 2019.
#rb none
#lockdown Nick.Penwarden

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: ben.marsh
#ROBOMERGE-SOURCE: CL 4662404 in //UE4/Main/...
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4662413 by ben marsh in Dev-Networking branch]
2018-12-14 13:44:01 -05:00
ryan gerleve
289a4fa159 Copying //UE4/Fortnite-Staging to Dev-Main (//UE4/Dev-Main)
#rb none
#lockdown Nick.Penwarden

#ROBOMERGE-OWNER: ryan.gerleve
#ROBOMERGE-AUTHOR: ben.marsh
#ROBOMERGE-SOURCE: CL 4567513 in //UE4/Main/...
#ROBOMERGE-BOT: ENGINE (Main -> Dev-Networking)

[CL 4573136 by ryan gerleve in Dev-Networking branch]
2018-11-16 14:01:06 -05:00
Ryan Gerleve
9af1fb3463 Merging //UE4/Dev-Main to Dev-Networking (//UE4/Dev-Networking)
#rb none
#rnx

[CL 4516466 by Ryan Gerleve in Dev-Networking branch]
2018-10-29 10:01:28 -04:00
Ryan Gerleve
e7612e89b2 Merging //UE4/Dev-Main to Dev-Networking (//UE4/Dev-Networking)
#rb none
#rnx

[CL 4341874 by Ryan Gerleve in Dev-Networking branch]
2018-09-04 14:32:41 -04:00
Ryan Gerleve
ac280888a0 Merging //UE4/Dev-Main to Dev-Networking (//UE4/Dev-Networking)
#rb none
#rnx

[CL 4175728 by Ryan Gerleve in Dev-Networking branch]
2018-06-29 20:48:58 -04:00