Commit Graph

1354 Commits

Author SHA1 Message Date
jack porter
63ef117561 Perform restricted staged folder name checking before doing platform filename remapping
Back out incorrect fix from CL 20056908
#jira UE-147094
#preflight 62790b2fea2a9db9dc560710
#rb Adam.Kinge
#lockdown Mitchell.Wilson

#ROBOMERGE-AUTHOR: jack.porter
#ROBOMERGE-SOURCE: CL 20110639 in //UE5/Release-5.0/... via CL 20111734
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20114350 by jack porter in ue5-main branch]
2022-05-09 19:52:57 -04:00
Devin Doucette
bd6866bfe9 BuildCMakeLib: Added support for targeting arm64 on Win64 with VS2019
This allows a library to use a Win64+arm64 build for HoloLens without providing a separate set of libs for HoloLens.

#preflight 62791db7066ebb49109d692f
#rb Zousar.Shaker

[CL 20106529 by Devin Doucette in ue5-main branch]
2022-05-09 13:57:58 -04:00
Dan Thompson
3a6321167b Packaging Reference Chunk Database - "Lossless Patch Preventer"
Can now provide reference iostore containers to reuse compressed chunks from. If a match is found on *the decompressed data*, instead of recompressing the blocks, they are read off of disk. This allows tweaks of the compressor algorithm without introducing changes as the runtime still sees the exact same data. Additionally this allows for fairly dramatic staging speedups as nvme speeds are significantly faster than high effort compressions. This is distinct from the DDC compression because:
1) DDC compression ties in the compressor version/method
2) We are explicitly interested in chunks that are deployed to end users, not merely cached for speed.

To facilitate this, several changes were made to IoStore:

FIoStoreReader now directly reads from IFileHandles* instead of routing through the GenericPlatformFile async read system, as that system is sensitive to build #defines and can result in constant file opens under load (indeed, for anything not a .pak file, every read is an open/close).

Cold file cache read speed improvements from ~140MB/s to ~1 GB/s. Hot is more.

Additionally:
    FIoStoreReader switched to UE::Tasks from taskgraph for tasks in order to facilitate task retraction during waits as the previous ReadAsync call was trivial to deadlock when called from worker threads due to its use of TFuture<>.
    FIoStoreReader::ReadCompressed now returns the compressed blocks as they were on disk - padded to AES encryption block size.

#rb fabian.giesen
#rb jeff.roberts
#preflight 627586dcf77c9c2b543d4d8b

[CL 20086673 by Dan Thompson in ue5-main branch]
2022-05-06 18:22:44 -04:00
jack porter
a5d55f3660 Use platform-remapped staging path when checking for restricted folder names
#jira UE-147094
#rb Ben Zeigler
#preflight 6273b716c04406dfea7e49b2
#lockdown Rolando.Caloca

#ROBOMERGE-AUTHOR: jack.porter
#ROBOMERGE-SOURCE: CL 20056908 in //UE5/Release-5.0/... via CL 20057485
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20061168 by jack porter in ue5-main branch]
2022-05-05 14:09:36 -04:00
Josh Adams
c756b51630 - FIxed an exception when NOT not using oiptional stage dir
#rb trivial
#preflight skip

[CL 20060045 by Josh Adams in ue5-main branch]
2022-05-05 13:13:18 -04:00
Josh Adams
defdfa230a - Added option for IoStore to write optional files (.o.ucas) to a different directory. This is used when making a a cooked editor as DLC against a client build (the client build has to make the optional editor data, but we do not want to ship the optional data with the client). The MakeCookedEditor script will then copy the files from that location into the editor's Stage directory (under Content/Paks)
#rb carl-magnus.nordin,francis.hurteau
#preflight 6273f8fd8d32cd80d8a8ecdb

[CL 20059504 by Josh Adams in ue5-main branch]
2022-05-05 12:28:59 -04:00
PJ Kack
5d0ce32acd UAT: For editor quick launch and packaging, only pass -iostore and -compressed when using -pak, and never pass -compressed to the cook commandlet (that feature has been removed).
#jira none
#rb carlmagnus.nordin
#rnx
#preflight 6273b3d45b05fb4f60112bbe

[CL 20056179 by PJ Kack in ue5-main branch]
2022-05-05 07:41:13 -04:00
PJ Kack
6b6bc48fc5 UAT: Add additional startup and BuildCookRun logging and timings
#jira none
#rb trivial
#rnx
#preflight 627379dda56dd762120e6e2e

[CL 20055418 by PJ Kack in ue5-main branch]
2022-05-05 03:40:01 -04:00
carlmagnus nordin
172ee292ee UAT: Fixed issue with not finding packagestore.manifest when staging plugins from project folders not directly under the workspace root.
#jira UE-150990
#rb pk.kack
#preflight 6272315e4c664a1624631afc
#lockdown simon.tourangeau

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 20040973 in //UE5/Release-5.0/... via CL 20045721
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20047963 by carlmagnus nordin in ue5-main branch]
2022-05-04 16:14:10 -04:00
PJ Kack
65391545db UAT: Restrict the COTF -filehostip list to ip v4 host addresses and only include 127.0.0.1 on host platforms or platforms that support port redirection.
Makes connecting to cotf faster on some platforms.
Removes warnings like: LogSockets: Warning: Could not serialize fe80::edc3:3284:5b1b:1f32%7, got error code SE_NO_ERROR [0]
Makes the string shorter and more readable.

Test:
RunUAT.bat BuildCookRun -project=Samples\Games\ShooterGame\ShooterGame.uproject -platform=XYZ -config=development -build -cookonthefly -stage -run

FileHostIp example on PC & android:
Before: -filehostip=127.0.0.1+fe80::edc3:3284:5b1b:1f32%7+10.24.101.44+::1+127.0.0.1+fe80::58d4:b8b3:ff7a:4974%9+172.18.240.1
After : -filehostip=127.0.0.1+10.24.101.44+172.18.240.1

FileHostIp example on console:
Before: -filehostip=127.0.0.1+fe80::edc3:3284:5b1b:1f32%7+10.24.101.44+::1+127.0.0.1+fe80::58d4:b8b3:ff7a:4974%9+172.18.240.1
After : -filehostip=10.24.101.44+172.18.240.1

#jira none
#rb carlmagnus.nordin, per.larsson
#rnx
#preflight 62722ddafe45c124493258a8

[CL 20038819 by PJ Kack in ue5-main branch]
2022-05-04 04:18:17 -04:00
Catalin Dragoiu
72cb56da0b Allow programs to be built using the BuildTarget command.
#rb Andrew.Grant
#preflight 6270e09c5e6ce673f42e063c
#jira UE-149689

[CL 20026154 by Catalin Dragoiu in ue5-main branch]
2022-05-03 11:16:52 -04:00
Francis Hurteau
c719a4680a Add commandline boilerplate to support optional cooking through UAT
#rb Matt.Peters
#jira UE-142231
#preflight none

[CL 19929815 by Francis Hurteau in ue5-main branch]
2022-04-26 17:52:04 -04:00
Devin Doucette
565a2935c6 BuildCMakeLib: Added support for targeting the Unix platform group
#preflight 6261c9676119a1a496c533d6
#rb Zousar.Shaker

[CL 19863796 by Devin Doucette in ue5-main branch]
2022-04-22 08:56:28 -04:00
jonathan adamczewski
06aab1ddc7 BuildCommonTools: Don't try to build a non-existent library
#jira UE-149138
#trival
#preflight none
#rnx

[CL 19821497 by jonathan adamczewski in ue5-main branch]
2022-04-19 19:15:47 -04:00
axel riffard
6ab54619ca Add Harfbuzz / ICU support for tvOS
#jira UE-113408
#rb jack.porter
#preflight 19783834

[CL 19801919 by axel riffard in ue5-main branch]
2022-04-18 23:27:10 -04:00
josh adams
878a255ac2 - Added native mode for GenerateDSYM UAT script that will generate .dSYM files for Mac/IOS/TVOS projects. Main use is passing it project/target/config/etc params and it will read .target receipt file to figure out what executables and dylibs to generate .dSYMs for
- Disabled dSYM generation for Mac by default
#preflight 625d8c5c6813aa38e3d077f2
#rb axel.riffard
#jira none

[CL 19791055 by josh adams in ue5-main branch]
2022-04-18 14:37:50 -04:00
jonathan adamczewski
c12d99d854 .net6.0 is not supported in VS2019. Suppress warnings about that when using that IDE (with this change, when building UnrealBuildTool & AutomationTool)
#jira none
#rnx
#rb joe.kirchoff
#preflight 624f2c163c2aff69c54ff9eb

[CL 19675097 by jonathan adamczewski in ue5-main branch]
2022-04-07 15:06:02 -04:00
jonathan adamczewski
c1bb6901bf Upgrade to .NET 6.0
#jira UE-119846
#preflight 624cbb2ecc0872e96b0d5181

[CL 19652696 by jonathan adamczewski in ue5-main branch]
2022-04-06 14:46:50 -04:00
PJ Kack
5f6c6d5beb UAT: Add BuildCookRun support for running both a dedicated server and cotf server.
Merge RunClientWithServer() and RunStandAloneClient() functions into one RunClient() function to resolve numerous inconsistencies between them and between unattended/non-unattended mode.
Always use SC.StageTargetPlatform.RunClient() to launch clients so that the logic works on all platforms.
When either the client, dedicated server or cook server exits, then all other processes exits too.
Hitting Ctrl-c in UAT kills all processes.

ShooterGame testing on Win64, consoles and android:
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -cook -stage -pak -deploy -run
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -skipcook -skipstage -skipdeploy -run
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -skipcook -skipstage -skipdeploy -run -unattended
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -cookonthefly -stage -deploy -run
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -cookonthefly -stage -deploy -run -unattended
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -cookonthefly -zenstore -stage -deploy -run
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -cookonthefly -zenstore -stage -deploy -run -unattended

ShooterGame Win64 Desktop specific testing with lan server and multiple clients:
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -cookonthefly -zenstore -server -stage -run -map=Highrise -numclients=3
BuildCookRun -project=ShooterGame -platform=Win64 -config=development -skipbuild -cookonthefly -zenstore -server -stage -run -map=Highrise -numclients=3 -unattended

EngineTest testing on Win64 and console:
BuildCookRun -project=EngineTest -platform=Win64 -config=development -skipbuild -cook -stage -pak -deploy -run -runautomationtest=Project.System
BuildCookRun -project=EngineTest -platform=Win64 -config=development -skipbuild -cookonthefly -stage -deploy -run -runautomationtest=Project.System
BuildCookRun -project=EngineTest -platform=Win64 -config=development -skipbuild -cookonthefly -zenstore -stage -deploy -run -runautomationtest=Project.System

#jira none
#rb carlmagnus.nordin
#rnx
#preflight 624d79fdbf5b974989964f4c

[CL 19645492 by PJ Kack in ue5-main branch]
2022-04-06 08:03:43 -04:00
jonathan adamczewski
5053cae2d8 Launch UnrealBuildTool via dotnet unrealbuildtool.dll, not unrealbuildtool.exe
#jira none
#rnx
#preflight 624261bdb6084b98321e9251

[CL 19544939 by jonathan adamczewski in ue5-main branch]
2022-03-29 13:20:46 -04:00
jonathan adamczewski
0a6164bbd3 nullability annotations, rethrow fixes, net6.0 warning suppression
#jira none
#trivial
#rnx
#preflight 62425155c61d8a458f230657

[CL 19536049 by jonathan adamczewski in ue5-main branch]
2022-03-28 20:50:06 -04:00
jonathan adamczewski
5c68d1d9ef Revert //UE5/Main/... changelist 19531725 - return to NET Core 3.1
#preflight 62422fd0470aff98e946bfce
#jira UE-119846

[CL 19534572 by jonathan adamczewski in ue5-main branch]
2022-03-28 18:25:19 -04:00
jonathan adamczewski
18ac15cbb1 Upgrade to .NET 6.0
#jira UE-119846
#preflight 6241fc06470aff98e943c035

[CL 19531725 by jonathan adamczewski in ue5-main branch]
2022-03-28 15:30:56 -04:00
Johan Berg
f8d3bd1276 UnrealTraceServer should start with "fork" argument.
#rb martin.ridgers
#jira UE-146759
#preflight 623ae6d8bc1cf280387442fc

[CL 19477188 by Johan Berg in ue5-main branch]
2022-03-23 05:27:27 -04:00
David Harvey
4741cfabfd minor fix for CreatePlatformExtension if the .cs class name is does not have whitespace before the parent declaration.
#jira none
#rnx
#rb trivial
#preflight 6230bcc5fc4d442f89317669

[CL 19387468 by David Harvey in ue5-main branch]
2022-03-15 12:25:04 -04:00