- If a corrupt index is detected, do a retry but collect and log more information along the way to help diagnose the issue.
#rb stefan.boberg
#ROBOMERGE-SOURCE: CL 6685180 via CL 6685186 via CL 6751170
#ROBOMERGE-BOT: (v365-6733468)
[CL 6751664 by graeme thornton in Main branch]
Added new content only build node and now generates pak patch files for pak files not in original build.
Also merges manifest with original build
Also posts build to gamedev and prod
GenerateChunkDBs now takes in two additional parameters specifying how many rollback labels to process and which ones to optimize (preparation for future change where we process more chunkdbs).
Added support for generating buildinfo.ini for ps4.
#ROBOMERGE-SOURCE: CL 6278253 via CL 6287210
[CL 6287835 by daniel lamb in Main branch]
Profiling in suggests it improves streaming performance, and the paks are smaller
#rb none
[FYI] hongyi.yu
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 5810432 via CL 5810437 via CL 5810648 via CL 5813653 via CL 5813762
#ROBOMERGE-BOT: CORE (Main -> Dev-Core)
[CL 5843937 by ben woodhouse in Dev-Core branch]
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 5772400 via CL 5795133 via CL 5795152 via CL 5795156 via CL 5799600 via CL 5799704
#ROBOMERGE-BOT: CORE (Main -> Dev-Core)
[CL 5818691 by ben woodhouse in Dev-Core branch]
Profiling in suggests it improves streaming performance, and the paks are smaller
#rb none
[FYI] hongyi.yu
#ROBOMERGE-SOURCE: CL 5810432 via CL 5810437 via CL 5810648 via CL 5813653
[CL 5813762 by ben woodhouse in Main branch]
#jira UE-72242
[FYI] marc.audy
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 5708883 via CL 5708886 via CL 5708887 via CL 5708891 via CL 5709641 via CL 5709794
#ROBOMERGE-BOT: CORE (Main -> Dev-Core)
[CL 5718872 by ben woodhouse in Dev-Core branch]
#jira UE-72242
[FYI] marc.audy
#ROBOMERGE-SOURCE: CL 5708883 via CL 5708886 via CL 5708887 via CL 5708891 via CL 5709641
[CL 5709794 by ben woodhouse in Main branch]
#test Unrealpak
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: daniel.lamb
#ROBOMERGE-SOURCE: CL 5691627 via CL 5691628 via CL 5691630 via CL 5691631 via CL 5695674 via CL 5695740
#ROBOMERGE-BOT: CORE (Main -> Dev-Core)
[CL 5701481 by daniel lamb in Dev-Core branch]
- Add support for incremental gap filling. Does gap filling in multiple passes, allows us to limit patch size growth to some fixed percentage
- Fixed cooker order UEXP files (previously these were unordered, which could actually result in patches increasing in size after gap filling)
- Added better method for measuring fragmentation - reports as a percentage, takes order index into account
- Refactored OrderMapping via new FPakOrderMap class which knows about primary/secondary mappings
- Added various schemes for defragmentation, including OnePass method which matches legacy behaviour, and various incremental schemes. Incremental_PrimaryFirst seems to give best results
#rb hongyi.yu
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: ben.woodhouse
#ROBOMERGE-SOURCE: CL 5688530 via CL 5688536 via CL 5688538 via CL 5690113 via CL 5694330 via CL 5694501
#ROBOMERGE-BOT: CORE (Main -> Dev-Core)
[CL 5699633 by ben woodhouse in Dev-Core branch]
- Add support for incremental gap filling. Does gap filling in multiple passes, allows us to limit patch size growth to some fixed percentage
- Fixed cooker order UEXP files (previously these were unordered, which could actually result in patches increasing in size after gap filling)
- Added better method for measuring fragmentation - reports as a percentage, takes order index into account
- Refactored OrderMapping via new FPakOrderMap class which knows about primary/secondary mappings
- Added various schemes for defragmentation, including OnePass method which matches legacy behaviour, and various incremental schemes. Incremental_PrimaryFirst seems to give best results
#rb hongyi.yu
#ROBOMERGE-SOURCE: CL 5688530 via CL 5688536 via CL 5688538 via CL 5690113 via CL 5694330
[CL 5694501 by ben woodhouse in Main branch]
Reenabled oodle compression in 8.30 for android, ios and switch.
Cleaned up unrealpak commandline when run from UAT.
#rb Josh.Adams
[FYI] Ben.Marsh, Andrew.Grant
#ROBOMERGE-OWNER: robert.manuszewski
#ROBOMERGE-AUTHOR: daniel.lamb
#ROBOMERGE-SOURCE: CL 5619788 via CL 5619815 via CL 5619821 via CL 5619827 via CL 5623747 via CL 5623814
#ROBOMERGE-BOT: CORE (Main -> Dev-Core)
[CL 5628527 by robert manuszewski in Dev-Core branch]
Reenabled oodle compression in 8.30 for android, ios and switch.
Cleaned up unrealpak commandline when run from UAT.
#rb Josh.Adams
[FYI] Ben.Marsh, Andrew.Grant
#ROBOMERGE-SOURCE: CL 5619788 via CL 5619815 via CL 5619821 via CL 5619827 via CL 5623747
[CL 5623814 by daniel lamb in Main branch]
- Future plan would be not to use OpenSSL directly, but to go via PlatformCrypto or something like it which could provide platform specific RSA functionality if it exists
- Also contains some compensation code for platforms that are still stuck on an older version of OpenSSL (everything except windows)
- On platforms that use OpenSSL < 1.1.1, register a locking callback so we can provide mutexes and stop it crashing horribly when the same RSA key is used concurrently on different threads
Changed default key size in the CryptoKeys plugin to 4096 bits.
#jira UE-71377
#rb ian.fox, robert.manuszewski, ryan.gerleve
#ROBOMERGE-OWNER: graeme.thornton
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5447433 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5447485 by graeme thornton in Main branch]
- CryptoKeys plugin can now be told how many bits to use for keys. Defaults to 2048 as a happy medium between security and performance
- After creating the OpenSSL RSA key, use the exp/mod values directly rather than recomputing from primes
- Added RSA.h with some TBigInt based helpers for RSA keys
- Added FPakSignatureFile to wrap the contents of the pak .sig file, including determining that it is of a new format that supports versioning. Wraps the encryption and decryption of he master table hash
- Changed pak master signature hash from crc to SHA1
- Modified access to executable embedded public key data so that it is just an array of bytes. Will allow custom key sizes in the future.
#rb joe.barrett, stefan.boberg
#jira UE-71280, UE-71377
#ROBOMERGE-OWNER: graeme.thornton
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5381335 in //UE4/Release-4.22/...
#ROBOMERGE-BOT: RELEASE (Release-4.22 -> Main)
[CL 5383057 by graeme thornton in Main branch]
- Added a "-sign" parameter to enable signing for the given pak. The cached settings in the crypto.json file aren't used to enable signing any more.
- Encapsulated all signing and encryption keys in a single keychain object which is passed around, rather than individual keys
- Added some validation to UBT/UAT encryption config parsing so that if you enable any signing or encryption stuff but haven't generated a key, it'll warn and disable everything.
#jira
#ROBOMERGE-OWNER: graeme.thornton
#ROBOMERGE-AUTHOR: graeme.thornton
#ROBOMERGE-SOURCE: CL 5237725 via CL 5237726 via CL 5240337
[CL 5240343 by graeme thornton in Main branch]
- Brings over the necessary engine changes for embedding UE4 mobile as a dylib/so in native mobile app
- Various changes for facial animation, screen recording, others
- ARKit and ARCore plugins were removed, as deemed "not ready"
#rb many people
#ROBOMERGE-OWNER: josh.adams
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 5201138 via CL 5203024
[CL 5226277 by Josh Adams in Main branch]
#rb gil.bribb
[FYI] gil.gribb
#ROBOMERGE-OWNER: josh.adams
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 4998359 via CL 4998360 via CL 4999252
[CL 5001374 by josh adams in Main branch]
#rb josh.adams
#ROBOMERGE-OWNER: josh.adams
#ROBOMERGE-AUTHOR: gil.gribb
#ROBOMERGE-SOURCE: CL 4993025 via CL 4993026 via CL 4995423
[CL 4999718 by josh adams in Main branch]
- -nommio on the commandline will disble it for testing
#rb gil.gribb
#ROBOMERGE-OWNER: josh.adams
#ROBOMERGE-AUTHOR: josh.adams
#ROBOMERGE-SOURCE: CL 4963115 via CL 4963118 via CL 4963812
[CL 4969283 by josh adams in Main branch]
