Commit Graph

3672 Commits

Author SHA1 Message Date
Chris Babcock
7e03d99b02 Fix overflow obb file installs for Android
#jira UE-151295
#android
#review @Jack.Porter
#rb Jack.Porter

#preflight https://horde.devtools.epicgames.com/job/6279d6d51217b3a23d17ab0a

[CL 20117619 by Chris Babcock in ue5-main branch]
2022-05-09 23:20:43 -04:00
jack porter
63ef117561 Perform restricted staged folder name checking before doing platform filename remapping
Back out incorrect fix from CL 20056908
#jira UE-147094
#preflight 62790b2fea2a9db9dc560710
#rb Adam.Kinge
#lockdown Mitchell.Wilson

#ROBOMERGE-AUTHOR: jack.porter
#ROBOMERGE-SOURCE: CL 20110639 in //UE5/Release-5.0/... via CL 20111734
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20114350 by jack porter in ue5-main branch]
2022-05-09 19:52:57 -04:00
Devin Doucette
bd6866bfe9 BuildCMakeLib: Added support for targeting arm64 on Win64 with VS2019
This allows a library to use a Win64+arm64 build for HoloLens without providing a separate set of libs for HoloLens.

#preflight 62791db7066ebb49109d692f
#rb Zousar.Shaker

[CL 20106529 by Devin Doucette in ue5-main branch]
2022-05-09 13:57:58 -04:00
Dan Thompson
3a6321167b Packaging Reference Chunk Database - "Lossless Patch Preventer"
Can now provide reference iostore containers to reuse compressed chunks from. If a match is found on *the decompressed data*, instead of recompressing the blocks, they are read off of disk. This allows tweaks of the compressor algorithm without introducing changes as the runtime still sees the exact same data. Additionally this allows for fairly dramatic staging speedups as nvme speeds are significantly faster than high effort compressions. This is distinct from the DDC compression because:
1) DDC compression ties in the compressor version/method
2) We are explicitly interested in chunks that are deployed to end users, not merely cached for speed.

To facilitate this, several changes were made to IoStore:

FIoStoreReader now directly reads from IFileHandles* instead of routing through the GenericPlatformFile async read system, as that system is sensitive to build #defines and can result in constant file opens under load (indeed, for anything not a .pak file, every read is an open/close).

Cold file cache read speed improvements from ~140MB/s to ~1 GB/s. Hot is more.

Additionally:
    FIoStoreReader switched to UE::Tasks from taskgraph for tasks in order to facilitate task retraction during waits as the previous ReadAsync call was trivial to deadlock when called from worker threads due to its use of TFuture<>.
    FIoStoreReader::ReadCompressed now returns the compressed blocks as they were on disk - padded to AES encryption block size.

#rb fabian.giesen
#rb jeff.roberts
#preflight 627586dcf77c9c2b543d4d8b

[CL 20086673 by Dan Thompson in ue5-main branch]
2022-05-06 18:22:44 -04:00
Ryan Durand
d18830bae2 Make FindTagNamesFromList static since it doesn't use any member variables and allows it to be called from other classes.
#rnx
#rb ryan.hummer
#jira none
#preflight

[CL 20080477 by Ryan Durand in ue5-main branch]
2022-05-06 14:07:08 -04:00
carl bystrom
283ad82702 BuildGraph: Add more parameters to Docker-Build task
Supporting BuildKit and .dockerignore files will help improve performance without resorting to temporary staging of files.
#preflight 6275287ed8373707f211ffa8

[CL 20075007 by carl bystrom in ue5-main branch]
2022-05-06 09:58:56 -04:00
Joe Conley
8d2a0ef148 #jira UE-147307 //UE5/Main - Stage EngineTest Game HoloLens - Failed to copy Manifest
HoloLensPlatform::GetPathToVCLibsPackages() needs to call MicrosoftPlatformSDKs::EnumerateSdkRootDirs() function and loop through the result trying them in order, rather than always looking for a hardcoded path.  Also adding VCLibs for arm64 necessary for HoloLens to CarefullyRedist.  For some reason, they are in Microsoft SDKs/Windows Kits instead of just in Windows Kits/, so I matched that directory structure in CarefullyRedist, then the code I'm adding the HoloLensPlatform.Automation.cs can use the same path to look for it if it's installed manually or from AutoSDKs.
#preflight 62742e6d4a2e9340486c9adf
#preflight 627430be03269096abd69fdf
testing: Quick Launch HoloLensQA project to HoloLens device still works!

[CL 20065655 by Joe Conley in ue5-main branch]
2022-05-05 18:05:18 -04:00
Josh Adams
f18143aa8b - Added missing commandline option for the optional file staging directory (-optionalfilestagingdirectory=)
#rb trivial
#preflight 627423bafd59d6606e0162f0

[CL 20063141 by Josh Adams in ue5-main branch]
2022-05-05 15:36:34 -04:00
jack porter
a5d55f3660 Use platform-remapped staging path when checking for restricted folder names
#jira UE-147094
#rb Ben Zeigler
#preflight 6273b716c04406dfea7e49b2
#lockdown Rolando.Caloca

#ROBOMERGE-AUTHOR: jack.porter
#ROBOMERGE-SOURCE: CL 20056908 in //UE5/Release-5.0/... via CL 20057485
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20061168 by jack porter in ue5-main branch]
2022-05-05 14:09:36 -04:00
Josh Adams
c756b51630 - FIxed an exception when NOT not using oiptional stage dir
#rb trivial
#preflight skip

[CL 20060045 by Josh Adams in ue5-main branch]
2022-05-05 13:13:18 -04:00
Josh Adams
defdfa230a - Added option for IoStore to write optional files (.o.ucas) to a different directory. This is used when making a a cooked editor as DLC against a client build (the client build has to make the optional editor data, but we do not want to ship the optional data with the client). The MakeCookedEditor script will then copy the files from that location into the editor's Stage directory (under Content/Paks)
#rb carl-magnus.nordin,francis.hurteau
#preflight 6273f8fd8d32cd80d8a8ecdb

[CL 20059504 by Josh Adams in ue5-main branch]
2022-05-05 12:28:59 -04:00
David Harvey
76d361df75 Low-Level test runner for windows-based platform extensions.
#jira UE-131865, UE-139998, UE-147194
#rnx
#preflight 6272becc365de864c49eefe8
#rb Eric.McDaniel, Chris.Constantinescu

[CL 20057912 by David Harvey in ue5-main branch]
2022-05-05 10:56:38 -04:00
PJ Kack
5d0ce32acd UAT: For editor quick launch and packaging, only pass -iostore and -compressed when using -pak, and never pass -compressed to the cook commandlet (that feature has been removed).
#jira none
#rb carlmagnus.nordin
#rnx
#preflight 6273b3d45b05fb4f60112bbe

[CL 20056179 by PJ Kack in ue5-main branch]
2022-05-05 07:41:13 -04:00
PJ Kack
6b6bc48fc5 UAT: Add additional startup and BuildCookRun logging and timings
#jira none
#rb trivial
#rnx
#preflight 627379dda56dd762120e6e2e

[CL 20055418 by PJ Kack in ue5-main branch]
2022-05-05 03:40:01 -04:00
Josh Adams
5bd689a00d - FIxed a Turnkey bug where flags weren't surrounded in quotes, so the editor wasn't parsting all of the flags properly
#jira none
#rb trivial
#preflight skip

[CL 20053659 by Josh Adams in ue5-main branch]
2022-05-04 20:43:36 -04:00
carlmagnus nordin
172ee292ee UAT: Fixed issue with not finding packagestore.manifest when staging plugins from project folders not directly under the workspace root.
#jira UE-150990
#rb pk.kack
#preflight 6272315e4c664a1624631afc
#lockdown simon.tourangeau

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 20040973 in //UE5/Release-5.0/... via CL 20045721
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20047963 by carlmagnus nordin in ue5-main branch]
2022-05-04 16:14:10 -04:00
Francis Hurteau
edddb2f628 Expose -editoroptional option to cooked editor makerelease
#rb Josh.Adams
#jira none
#preflight none

[CL 20047844 by Francis Hurteau in ue5-main branch]
2022-05-04 16:10:05 -04:00
Ryan Durand
c2a32ddc91 Couple of temp storage clean up changes to let multiple clean up jobs run less painfully.
* Flag folders as delete in progress to allow other clean up jobs to skip a folder and go to the next one.
* Delete from newest to oldest in an attempt to provide a buffer before multiple jobs will attempt to delete the same folder.

#rnx
#jira UE-150924
#rb ryan.hummer,bryan.johnson
#preflight

[CL 20047320 by Ryan Durand in ue5-main branch]
2022-05-04 15:44:17 -04:00
ben hoffman
c2aaa5ff8a Set a default UserName for the SteamDeck device confiuration. As of SteamOS version 3.1, "deck" is the required UserName to be used when making a remote connection to the device. Eventually it will be configurable, so we will allow users to set it via the config.
Also add a "SteamDeck" prefix to the default device name. This makes sure that its obvious what device this is until we can get a nice icon.

#jira none
#rb Brandon.Schaefer
#rnx
#preflight 6272d0b8365de864c4beeaa6

[CL 20046990 by ben hoffman in ue5-main branch]
2022-05-04 15:32:08 -04:00
PJ Kack
65391545db UAT: Restrict the COTF -filehostip list to ip v4 host addresses and only include 127.0.0.1 on host platforms or platforms that support port redirection.
Makes connecting to cotf faster on some platforms.
Removes warnings like: LogSockets: Warning: Could not serialize fe80::edc3:3284:5b1b:1f32%7, got error code SE_NO_ERROR [0]
Makes the string shorter and more readable.

Test:
RunUAT.bat BuildCookRun -project=Samples\Games\ShooterGame\ShooterGame.uproject -platform=XYZ -config=development -build -cookonthefly -stage -run

FileHostIp example on PC & android:
Before: -filehostip=127.0.0.1+fe80::edc3:3284:5b1b:1f32%7+10.24.101.44+::1+127.0.0.1+fe80::58d4:b8b3:ff7a:4974%9+172.18.240.1
After : -filehostip=127.0.0.1+10.24.101.44+172.18.240.1

FileHostIp example on console:
Before: -filehostip=127.0.0.1+fe80::edc3:3284:5b1b:1f32%7+10.24.101.44+::1+127.0.0.1+fe80::58d4:b8b3:ff7a:4974%9+172.18.240.1
After : -filehostip=10.24.101.44+172.18.240.1

#jira none
#rb carlmagnus.nordin, per.larsson
#rnx
#preflight 62722ddafe45c124493258a8

[CL 20038819 by PJ Kack in ue5-main branch]
2022-05-04 04:18:17 -04:00
Catalin Dragoiu
72cb56da0b Allow programs to be built using the BuildTarget command.
#rb Andrew.Grant
#preflight 6270e09c5e6ce673f42e063c
#jira UE-149689

[CL 20026154 by Catalin Dragoiu in ue5-main branch]
2022-05-03 11:16:52 -04:00
Jerome Delattre
546cbcbcdb Gauntlet - Support different build target configuration for editor with RunUnreal command.
#jira UE-150433
#rnx
#rb Chris.Constantinescu, Eric.Knapik
#preflight skip,NotTestedByPreflight

[CL 19968765 by Jerome Delattre in ue5-main branch]
2022-04-28 16:58:19 -04:00
Tim Smith
92be35bfde Removed temporary code that was causing UAT to always rebuild plugins.
#rb trivial
#rnx
#preflight 62699d0f2735a2c0ea8ef83a

[CL 19945588 by Tim Smith in ue5-main branch]
2022-04-27 16:05:00 -04:00
jonathan adamczewski
f515402a7a CsCompile: Don't use /restore when compiling with xbuild
#jira none
#fyi ryan.hummer
#rnx
#preflight 62697de92735a2c0ea8a153c

[CL 19943308 by jonathan adamczewski in ue5-main branch]
2022-04-27 14:04:02 -04:00
Matt Peters
b250769d03 CookedEditor staging: Fix warnings about missing generated map files - do not add the uncooked version of map files for generated files that do not exist uncooked.
#jira UE-142791
#rb Josh.Adams
#rnx
#preflight 62694ae9b710a00f18b2fa77

[CL 19938955 by Matt Peters in ue5-main branch]
2022-04-27 10:03:41 -04:00