Commit Graph

80 Commits

Author SHA1 Message Date
josh adams
6142689460 - Added compression methods to the pak dump outpout
#rb trivial

#ROBOMERGE-SOURCE: CL 12873061 via CL 12873196 via CL 12873273
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v681-12776863)

[CL 12873301 by josh adams in Main branch]
2020-04-16 17:12:56 -04:00
daniel lamb
53e963cf99 Fixed compile error in my last checkin.
#ROBOMERGE-SOURCE: CL 12758498 via CL 12759098 via CL 12759136 via CL 12759170
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v678-12743200)

[CL 12759193 by daniel lamb in Main branch]
2020-04-13 13:37:30 -04:00
daniel lamb
9c6f24761e Extracting pak files now supports multiple encryption keys
Graeme.Thornton

#ROBOMERGE-SOURCE: CL 12757780 via CL 12758878 via CL 12758909 via CL 12759078
#ROBOMERGE-BOT: RELEASE (Release-Engine-Staging -> Main) (v678-12743200)

[CL 12759155 by daniel lamb in Main branch]
2020-04-13 13:36:39 -04:00
josh adams
7141cbac93 - Added -Info option for UnrealPak to dump some header info about a pak file
#rb trivial
[FYI] daniel.lamb


#ROBOMERGE-SOURCE: CL 12336069 via CL 12336079
#ROBOMERGE-BOT: (v671-12333473)

[CL 12336086 by josh adams in Main branch]
2020-03-20 10:52:57 -04:00
Arciel Rekman
e970ef5ea2 ShaderLibrary sorted by file open order, with other fixes (FORT-265600).
- Each shadermap is now associated with one or more assets, and shaders are laid out according to the
  open order of those assets, if provided. If not provided, the original ordering is kept. The original ordering
  will not be always deterministic, particularly if shader compilation needs to happen, because jobs can get
  reordered during the compilation.

Other fixes:

- Fixed sorting within the shadermap not being deterministic (FCompareShaderPrimaryKey)
- Fixed Niagara shadermap duplicating Platform property and not initializing it properly.
- Removed (unused) support for Child shader libraries (would not be deterministic).

#rb Ben.Ingram, Rob.Krajcarski


#ROBOMERGE-OWNER: Arciel.Rekman
#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 12249260 via CL 12249262 via CL 12250173
#ROBOMERGE-BOT: (v668-12245121)

[CL 12250313 by Arciel Rekman in Main branch]
2020-03-18 15:35:20 -04:00
Matt Peters
34c1a8d018 Fix bad merge - restore diffs from revision 50. 51 (Pak file freezing) was intentionally reverted, but 50 (pak file system handle mounting of IoStores) was a mistake and should have been kept.
#rb trivial
#rnx

[CL 12123942 by Matt Peters in Main branch]
2020-03-11 08:32:05 -04:00
Matt Peters
40f55732ba #jira FORT-236167
[REVIEW]
#rb Graeme.Thornton
Optimize Pak file indices by computing the PathHashIndex in unrealpak rather than using strings at runtime.
This also removes the need to shrink the string indices for performance, which was triggering a bug due to unsynchronized multithreaded access.

#ROBOMERGE-SOURCE: CL 11442058 in //Fortnite/Release-12.00/... via CL 11442063 via CL 11442073 via CL 11442081

[CL 11443295 by Matt Peters in Main branch]
2020-02-14 09:53:05 -05:00
Josh Adams
aa9705149b Copying Private-LoadTimes-4.24 stream to Main. Biggest changes are in Materials/Shader memory freezing.
#rb none

[CL 11282608 by Josh Adams in Main branch]
2020-02-06 13:13:41 -05:00
Stefan Boberg
55a8b15f1f Copying //UE4/Dev-Core to Main (//UE4/Main)
#rb none

[CL 11258477 by Stefan Boberg in Main branch]
2020-02-05 14:26:36 -05:00
Devin Doucette
070f44a07a Add debug context to DDC requests that take the key as a parameter
The existing GetSynchronous, GetAsynchronous, and Put functions are deprecated in favor of the DebugContext overloads. It is unlikely that a licensee has derived from FDerivedDataCacheInterface, but anyone who does will have a compile error that they override a non-virtual function and that they don't implement a pure virtual function, which will be straightforward to fix.

#rb Zousar.Shaker

#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 11210386 via CL 11210387
#ROBOMERGE-BOT: (v643-11205221)

[CL 11210609 by Devin Doucette in Main branch]
2020-02-03 17:17:56 -05:00
PJ Kack
ef8a7b620e AUTO MERGE
Merging //UE4/Main @ 11112898 to //UE4/Dev-Core (dev-core-l0178)

#rb none

(ushell-p4-mergedown)

[CL 11113593 by PJ Kack in Dev-Core branch]
2020-01-26 11:35:42 -05:00
graeme thornton
b445f78774 unrealpak: Fix a typo where I accidentally removed the last digit of a constant
#rnx


#ROBOMERGE-SOURCE: CL 11078276 via CL 11078288 via CL 11078295
#ROBOMERGE-BOT: (v637-11041722)

[CL 11078297 by graeme thornton in Main branch]
2020-01-21 18:09:35 -05:00
CarlMagnus Nordin
6abf1ca5c9 Create one IoDispatcher container file per pak file and let the pak file system handle mounting of IoStores
#rb per.larsson

[CL 11027862 by CarlMagnus Nordin in Dev-Core branch]
2020-01-16 06:01:45 -05:00
ryan durand
471d972e62 Updating copyright for Engine Developer.
#rnx
#rb none


#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)

[CL 10870584 by ryan durand in Main branch]
2019-12-26 15:32:37 -05:00
marc audy
25ee46da1d Fix error: copying variable of type 'TAtomic<int64>' (aka 'TAtomic<long long>') invokes deleted constructor
#jira
#rnx
[FYI] Graeme.Thornton


#ROBOMERGE-SOURCE: CL 10808963 via CL 10808964
#ROBOMERGE-BOT: (v610-10636431)

[CL 10808966 by marc audy in Main branch]
2019-12-19 09:13:55 -05:00
graeme thornton
180e80ba64 Added an option to unrealpak which forces all compressed files inside a pak to be compressed, even if the compression results in a larger file size
#rb per.larsson


#ROBOMERGE-SOURCE: CL 10808714 via CL 10808715
#ROBOMERGE-BOT: (v610-10636431)

[CL 10808716 by graeme thornton in Main branch]
2019-12-19 08:20:24 -05:00
marc audy
dbcf07a558 Use new inline versions of substring functions
Fix up nearby cases where ESearchCase::CaseSensitive should have been used
#jira
#rnx
#rb

#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: marc.audy
#ROBOMERGE-SOURCE: CL 10309793 via CL 10309818
#ROBOMERGE-BOT: (v593-10286020)

[CL 10309932 by marc audy in Main branch]
2019-11-19 13:27:07 -05:00
hongyi yu
b22caf3c66 Handle UInt64 output properly when list file with unrealpak
#ROBOMERGE-SOURCE: CL 9421816 via CL 9422754
#ROBOMERGE-BOT: (v480-9420520)

[CL 9423096 by hongyi yu in Main branch]
2019-10-04 16:50:41 -04:00
Stefan Boberg
1f813eb516 Copying //UE4/Dev-Core to Main (//UE4/Main)
#rb many

[CL 9405827 by Stefan Boberg in Main branch]
2019-10-03 16:26:48 -04:00
hongyi yu
d5615ff31e Add the option to force aligning files in pak files that are larger than block size and enable it
Aligning large files will significantly reduce patch size when doing delta patching on XboxOne.
However, it will also increase package size by about 5%.

#rb Daniel.Lamb


#ROBOMERGE-SOURCE: CL 8334773 via CL 8357932
#ROBOMERGE-BOT: (v401-8057353)

[CL 8359105 by hongyi yu in Main branch]
2019-08-27 21:23:37 -04:00
ben marsh
c60b615a40 Fix static analysis warning.
#rb none
[FYI] daniel.lamb
#jira


#ROBOMERGE-SOURCE: CL 8288568 via CL 8288569 via CL 8288836
#ROBOMERGE-BOT: (v401-8057353)

[CL 8288864 by ben marsh in Main branch]
2019-08-25 16:33:56 -04:00
daniel lamb
20429a57b7 Fixed alignment of non compressed files in pak files.
Also removed DDC config file.  Will need to figure out another way to do this when we enable DDC.
Graeme.Thornton


#ROBOMERGE-SOURCE: CL 8240468 via CL 8240483 via CL 8243394
#ROBOMERGE-BOT: (v401-8057353)

[CL 8243494 by daniel lamb in Main branch]
2019-08-23 19:57:51 -04:00
daniel lamb
cee443cc76 Fixed crash in unrealpak in uncompressed non async code path.
#ROBOMERGE-SOURCE: CL 8223513 via CL 8223514 via CL 8229913
#ROBOMERGE-BOT: (v401-8057353)

[CL 8230750 by daniel lamb in Main branch]
2019-08-23 12:04:28 -04:00
bob tellez
696d945a82 Merging CL#8215904 from //Fortnite/Release-10.30
Multithreaded file compression in unrealpak CreatePak.  Local testing shows 50% improvement.
Added compressed files to the DDC this doesn't improve performance when using Oodle Optimal2 but does improve when using Oodle Optimal3.

#test preflight switch builds.
Graeme.Thornton


#ROBOMERGE-AUTHOR: Daniel.Lamb
#ROBOMERGE-SOURCE: CL 8216871 via CL 8229351
#ROBOMERGE-BOT: (v401-8057353)

[CL 8230264 by bob tellez in Main branch]
2019-08-23 11:56:05 -04:00
daniel lamb
e59eda4aec Added validation to patch generation to detect when unsupported files are changed.
#ROBOMERGE-SOURCE: CL 7937403 via CL 7937406 via CL 7937407 via CL 7940721
#ROBOMERGE-BOT: (v391-7919777)

[CL 7940816 by daniel lamb in Main branch]
2019-08-09 20:38:31 -04:00