Commit Graph

200 Commits

Author SHA1 Message Date
carlmagnus nordin
ffd1e2abfb ZenCook: Use the oplog for package store entries instead of writing a container header
#preflight 63720da8ee4d25f90aacf4dd
#rb per.larsson

[CL 23116204 by carlmagnus nordin in ue5-main branch]
2022-11-14 06:35:53 -05:00
henrik karlsson
8f895fef35 Added includes needed after removing includes in headers
#preflight 6360b63e41625be270a6e464
#rb none

[CL 22888775 by henrik karlsson in ue5-main branch]
2022-11-01 15:14:15 -04:00
dan engelbrecht
2f8823f7dc Pass along path to project file when zen server is running locally so we can detect when it is removed and garbage-collect any projectstore entries on the zen side.
Allow Cook to remote Zen, if we detect that we are targetting a zenservice on a different machine we change the local file references to attachment based references.
#rb stefan.boberg
#preflight 63491c31ad0f7e2f20230b39

[CL 22525021 by dan engelbrecht in ue5-main branch]
2022-10-14 08:17:43 -04:00
arciel rekman
1ad6c3648e Fix Lyra crash due to shader decompression failure (UE-165785).
- PCD3D_SM5 and PCD3D_SM6 had a unique situation of having identical (bytecode-wise and output hash-wise) shaders (raytracing is allowed in PCD3D_SM5). This resulted in those IoStore chunks being deduplicated, however, the actual contents were not identical due to optional data (which isn't included in the shader hash, see UE-166389).
- This fix makes sure we don't use just the shader hash alone to produce IoStore chunk hashes. Hash will now include the shader size and the shader format name.
- A few early guards added here and there to detect the problem early.

Also fixed a memory leak in UnrealPak pointed by CM.

#rb [at]Yuriy.ODonnell, [at]CarlMagnus.Nordin, [at]Christopher.Waters, [at]Jason.Nadro
#jira UE-165785
#preflight 633f3beed718720d2d58f37e

[CL 22402802 by arciel rekman in ue5-main branch]
2022-10-07 12:32:35 -04:00
danny couture
933197ef5b Fix TSAN compilation in editor for some TSAN_ATOMIC globals using TGuard scopes
#rb Devin.Doucette
#preflight 633f95992a0a2c1ead572ad6

[CL 22401373 by danny couture in ue5-main branch]
2022-10-07 11:14:15 -04:00
dan engelbrecht
502ded405a Make sure we wait for any async commits in ZenStoreWriter to complete before sending EndCook.
Previous implementation only waited for the commit to be picked up by the worker thread.
Simplified CommitQueue event logic.
#rb zousar.shaker per.larsson
#preflight none

[CL 22364055 by dan engelbrecht in ue5-main branch]
2022-10-05 17:41:51 -04:00
David Harvey
585d290ca0 remove warning about missing utoc files when generating patches.
- the default ordering is deterministic based on which packages are being imported so the warning is not appropriate for all circumstances.

#jira UE-164729
#rb CarlMagnus.Nordin
#rnx
#preflight 632c447b100305080652a7c0

[CL 22134373 by David Harvey in ue5-main branch]
2022-09-22 07:24:13 -04:00
carlmagnus nordin
83314ca92c IoStore: Downgraded failure to obtain package name from file name to a warning
#rb per.larsson
#preflight 632acaf60a4ad044deef90d5

[CL 22112911 by carlmagnus nordin in ue5-main branch]
2022-09-21 10:45:52 -04:00
Dan Thompson
1648e152b4 Fix passing -WriteBackMetadataToAssetRegistry via RunUAT. Now you can pass this in ushell after the -- when doing .stage game.
#rb martin.ridgers
#preflight 6320b01c506f1a33e05a6765

[CL 21986745 by Dan Thompson in ue5-main branch]
2022-09-13 13:21:49 -04:00
robert millar
0491bcc13c FNames containing asset paths are deprecated. FSoftObjectPath or FTopLevelAssetPath should be used instead.
Fixups for misc developer dir files.

#jira UE-161932
#rb none
#preflight 631f8070fd6da68d68b56472

[CL 21976423 by robert millar in ue5-main branch]
2022-09-12 19:53:59 -04:00
Dan Thompson
039bb04ac7 Log AR load errors (turns out, not actually logged when it's missing on disk!)
#rb trivial
#preflight 6305511cae13a5a098fea3e4

[CL 21522835 by Dan Thompson in ue5-main branch]
2022-08-23 18:22:09 -04:00
mark lintott
3d47f895e5 Fixed unacceptable words in PakFileUtilities
#jira UE-158611
#rb pj.kack
#preflight 62f383dbbc175ec68c338e38

[CL 21316108 by mark lintott in ue5-main branch]
2022-08-10 06:24:40 -04:00
bryan sefcik
de1956f47b Ran IWYU on Public headers under Engine/Source/Developer/...
Headers are updated to contain any missing #includes needed to compile and #includes are sorted.  Nothing is removed.

#ushell-cherrypick of 21064294 by bryan.sefcik
#jira
#preflight 62d5c2111062f2e63015e598

#ROBOMERGE-OWNER: bryan.sefcik
#ROBOMERGE-AUTHOR: bryan.sefcik
#ROBOMERGE-SOURCE: CL 21155249 via CL 21158121 via CL 21161259
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v972-20964824)

[CL 21182053 by bryan sefcik in ue5-main branch]
2022-07-20 12:03:45 -04:00
David Harvey
cceef86205 warn if the previous container files could not be found when creating a new release version - otherwise the patch delta can be significantly higher on some platforms.
#jira UE-132761
#rnx
#rb CarlMagnus.Nordin
#preflight 62bedd9ab447118888501bc2

[CL 20913019 by David Harvey in ue5-main branch]
2022-07-01 07:47:10 -04:00
CarlMagnus Nordin
5c69443c62 IoStore:
Default to using the current build crypto keys when reading previous builds for layout matching
Fixed incorrect order of previous build chunks when layout matching against a previous build
#rb david.harvey
#preflight 62bd9ea0a3568e30663af6fa

[CL 20895985 by CarlMagnus Nordin in ue5-main branch]
2022-06-30 11:15:16 -04:00
Matt Peters
ec493e4052 Multiprocess Cook - Add CookWorkerMode and IWorkerRequests interface.
#rb Zousar.Shaker
#rn Cooking, Minor
#preflight 62b4ff53970a8db3373f6e3c
#robomerge EngineMerge

[CL 20807998 by Matt Peters in ue5-main branch]
2022-06-24 08:14:11 -04:00
carlmagnus nordin
bc0b9cf56d IoStore: Add support for staging and loading auto optional packages
#preflight 62a879e8d502a5543671efa4
#rb francis.hurteau

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 20699731 via CL 20699906 via CL 20699913 via CL 20699917
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v968-20684695)

[CL 20700279 by carlmagnus nordin in ue5-main branch]
2022-06-17 04:25:23 -04:00
carlmagnus nordin
c12d76ebe2 IoStore: Removed warning about missing packages not getting assigned a load order
#preflight 62a087f7232daff7b3d5ab53
#rb pj.kack

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 20572957 via CL 20573494 via CL 20573503 via CL 20573508
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v954-20466795)

[CL 20573902 by carlmagnus nordin in ue5-main branch]
2022-06-09 07:37:49 -04:00
Devin Doucette
ffb60052c5 Removed runtime dependencies on DerivedDataCache
#preflight 6298f3250f6e137393b9dea7
#rb Zousar.Shaker
#rnx

[CL 20472431 by Devin Doucette in ue5-main branch]
2022-06-02 13:44:07 -04:00
Zousar Shaker
d34d74fad2 Avoid compiling in desktop-only Zen functionality for Stadia due to the use of PLATORM_UNIX instead of PLATFORM_LINUX.
#rb devin.doucette
#fyi martin.ridgers
#preflight 62967ef3261e39f6206b0a82

[CL 20444919 by Zousar Shaker in ue5-main branch]
2022-05-31 20:12:27 -04:00
Zousar Shaker
edf8ae1f0b 2nd submit:
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath.  This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store.  Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.

Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298
#preflight 629642a1be0ae0b33a46d012

[CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
Zousar Shaker
77f51a02b9 [Backout] - CL20433163
#fyi Zousar.Shaker
Original CL Desc
-----------------------------------------------------------------
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath.  This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store.  Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.

Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298

[CL 20433363 by Zousar Shaker in ue5-main branch]
2022-05-30 19:32:59 -04:00
Zousar Shaker
2f6cbf5012 Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not.
Tested Zen operation using ShooterGame editor.
Tested Jupiter operation using CitySample editor (empty local cache).

#rb devin.doucette
#fyi stefan.boberg
#preflight 62953e65401169f78c57c298

[CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
devin doucette
b2a07ea03e DDC: Merge from UE5/Main
#preflight 6288ff678828ea88c8af7034
#preflight 628ab5d93246d5019db76ed2
#rb none
#rnx

#ROBOMERGE-OWNER: devin.doucette
#ROBOMERGE-AUTHOR: Devin.Doucette
#ROBOMERGE-SOURCE: CL 20353148 via CL 20353832 via CL 20353839
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v948-20297126)

[CL 20355348 by devin doucette in ue5-main branch]
2022-05-24 16:40:25 -04:00
CarlMagnus Nordin
210d67b30e Promoted Zen project id generation to FApp
#rb pj.kack
#preflight 628769032c34da686ac82442

[CL 20316798 by CarlMagnus Nordin in ue5-main branch]
2022-05-23 01:49:25 -04:00