Commit Graph

1454 Commits

Author SHA1 Message Date
james doverspike
4e78faf0de Cosmetic bulk asset streaming
Adds support for marking skeletal meshes and textures as optional files that stream from the cloud, typically used for cosmetics. Meshes and textures will now upload all of their non-inlined LODs as optional bulk files to the cloud, removing them from the pak files. The client will download into the VirtualFileCache in PersistentDownloadDir/VFC, which evicts least recently used LODs. These assets will have a longer delay to update to the higher quality LOD when their LODs are not in cache because they are now downloaded on-demand.

[CL 24251508 by james doverspike in ue5-main branch]
2023-02-16 01:47:06 -05:00
james singer
85828bb4a6 Fix for installed builds on linux not having a stable UUID from machine to machine
Note: Since Linux has no launcher, EnumerateLauncherEngineInstallations doesn't take care of this for us in Epic created Linux installed builds

#rb @Brandon.Schaefer, @Zack.Neyland
#jira UE-171276
#preflight 63ed495f5c7bd278c1fd149e

[CL 24241578 by james singer in ue5-main branch]
2023-02-15 16:20:13 -05:00
FineRedMist
a2f83cd38c PR #10116: Add -ubtargs option to MegaXGE to pass to UnrealBuildTool (Contributed by FineRedMist)
#preflight 63ebda77dd72a8de9e4abee8

[CL 24217629 by FineRedMist in ue5-main branch]
2023-02-14 14:20:09 -05:00
carlmagnus nordin
50ef48b12c Fixed issue with ucas and utoc files not being copied to the output directory when creating chunk installs
#jira UE-172550
#rb pj.kack
#preflight 63eb4061346bca00357d7ac8

[CL 24213731 by carlmagnus nordin in ue5-main branch]
2023-02-14 12:03:03 -05:00
hilda cruz
fd186eaf71 [Backout] - CL24206643
[FYI] per.larsson
Original CL Desc
-----------------------------------------------------------------
This change adds support for defining ondemand pak file rule(s). These rules
overrides existing rules created from asset chunking and is currenlty only meant to be used for .ubulk/.uptnl files.
Ondemand content is serialized to loose files with a new I/O store writer to /MyGame/ContentOnDemand/Paks/{MyPakRuleName}.

#rb CarlMagnus.Nordin,PJ.Kack
#preflight 63e39130786751d1e010c14c
#rnx

[CL 24213673 by hilda cruz in ue5-main branch]
2023-02-14 12:01:56 -05:00
per larsson
e7c2bdc3e8 This change adds support for defining ondemand pak file rule(s). These rules
overrides existing rules created from asset chunking and is currenlty only meant to be used for .ubulk/.uptnl files.
Ondemand content is serialized to loose files with a new I/O store writer to /MyGame/ContentOnDemand/Paks/{MyPakRuleName}.

#rb CarlMagnus.Nordin,PJ.Kack
#preflight 63e39130786751d1e010c14c
#rnx

[CL 24213660 by per larsson in ue5-main branch]
2023-02-14 12:01:42 -05:00
per larsson
373dd67228 Ondemand PakFile rules
This change adds support for defining ondemand pak file rule(s). These rules
overrides existing rules created from asset chunking and is currenlty only meant to be used for .ubulk/.uptnl files.
Ondemand content is serialized to loose files with a new I/O store writer to /MyGame/ContentOnDemand/Paks/{MyPakRuleName}.

#rb CarlMagnus.Nordin,PJ.Kack
#preflight 63e39130786751d1e010c14c
#rnx

[CL 24096114 by per larsson in ue5-main branch]
2023-02-09 08:21:15 -05:00
dan thompson
4a6b8c8b00 Removal of the r.Shaders.CompressionFormat CVar, and shader group compression during UnrealPak now inherits the packaging compresison level.
Due to the fact that UnrealPak doesn't load INIs, the cvar was always its default value. This, combined with the fact that shaders don't store what codec they are compressed with means that setting this cvar to anything other than default (oodle) causes a crash in UnrealPak as DecompressShader will try to decompress with Oodle independent of the actual codec. Rather than plumb this information down through ProjectParams, I felt it prudent to just always use Oodle.

Additionally, in order to facilitate faster iteration times for development build farm jobs, the shader compression level used during UnrealPak will inherit from the overall package compression level set via Project Settings (PackageCompressionLevel_*).

This tacitly avoids patch issues as the default shader compression level happens to be the default package compression level, except now it inherits any faster levels for iteration time.

Note that we still use Mermaid as the codec as it's faster for decompressing than Kraken, which we care about for shaders.

#rb Fabian.Giesen
#rb Arciel.Rekman
#preflight 63e2f94c786751d1e0ffb290

[CL 24092953 by dan thompson in ue5-main branch]
2023-02-09 00:35:55 -05:00
joe pribele
759bb53070 adding warning to runaut buildplugin when HostPlatforms contains unsupported platforms
https://p4-swarm.epicgames.net/reviews/23960551

#jira https://jira.it.epicgames.com/browse/UE-167403
#rb zousar.shaker
#preflight 63dbe11e797b029c0adb4219

[CL 23970784 by joe pribele in ue5-main branch]
2023-02-02 12:32:10 -05:00
axel riffard
f0898e1c0d Remote Prepare to Debug
#jira UE-120589
#rb josh.adams
#preflight 63db32457a39a18021af8c38

[CL 23964201 by axel riffard in ue5-main branch]
2023-02-01 23:02:14 -05:00
Devin Doucette
fceb1519a7 BuildCMakeLib: Added [DoesNotNeedP4CL] to reduce startup time
#preflight 63da9ed9cf52968117248bcf
#rb Joe.Pribele, Matt.Peters
#rnx

[CL 23952225 by Devin Doucette in ue5-main branch]
2023-02-01 12:58:33 -05:00
Josh Adams
2c989f324c - Set default Mac architecture to universal (arm64+x64)
- Refactored Android to match with Mac way of setting default
#rb chris.babcock,zack.neyland
#preflight 63d2f355be1970f882bccca6

[CL 23876026 by Josh Adams in ue5-main branch]
2023-01-26 16:49:40 -05:00
Joe Kirchoff
f30e759f1b AutomationTool: Remove stale -iwyu UBT arg, whatvever it used to do is no longer available and this flag now runs the IWYU toolchain
#rnx
#rb Henrik.Karlsson
#preflight 63d1745c94644f3e8ee6e0c1

[CL 23852131 by Joe Kirchoff in ue5-main branch]
2023-01-25 13:30:30 -05:00
josh adams
1610c3bee3 UnrealArch/UnrealArchitectures changes
- Creates the UnrealArchitectures class, which wraps a list of UnrealArch objects
 - UnrealArch is a single architecture, expandable enum-like struct
 - There is no more concept of "no/default architecture", there is always a valid active architecture when building
 - Most uses of "string Architecture" are replaced with one of the two above, depending if multiple architectures are supported or not
 - UnrealArch has some platform-extensions for platform-specific naming (like Linux adds in LinuxName that turns, for instance, Arm64 -> aarch64-unknown-linux-gnueabi, which is used in folder names, etc)
 - UnrealArch has bIsX64 which can be used determine intel instruction set (as opposed to arm)
 - TargetRules class has an "Architecture" accessor that will return a single architecture if the active architectures is a single architecture, or throw an exception if multiple. This is useful in a majority of the cases where a paltform can only have a single architecture active in TargetRules (microsoft platforms, for instance, will create separate targets when compiling multiple architectures at once)
 - Added UnrealArchitectureConfig class, which contains all the architecture information for a platform (what architectures are supported, what ones are currently active for given project, etc)

#preflight 63c81fb5b065224750a1759e
#rb mike.fricker,roman.dzieciol,joe.kirchoff,dmytro.vovk,brandon.schaefer [various parts]
#p4v-preflight-copy 23562471

[CL 23829977 by josh adams in ue5-main branch]
2023-01-24 09:30:28 -05:00
PJ Kack
b1764482c1 UAT: Make sure to use the specified unreal executable instead of a hard coded development one when launching the cook on the fly server.
Fix bug where quick launching with cotf from a debug editor failed to launch the cook server due to a missing or old development editor executable.
Since the default executable is the the console -cmd.exe one, implement ShellExecute to open a separate shell for the server output, also fixes duplicated stdout lines in that window.
Improve BuildCookRun log messages for booting and shutting down server and client.

#jira UE-173897, UEENGQA-80955
#rb carlmagnus.nordin
#preflight 63ca9499c503023ab986b09c

[CL 23788926 by PJ Kack in ue5-main branch]
2023-01-20 08:38:42 -05:00
Tim Smith
8571b13a4a Removed the building of C++ UHT from UAT code
#rb joe.kirchoff
#preflight 63c8422dc246f148565e394b

[CL 23761148 by Tim Smith in ue5-main branch]
2023-01-18 14:11:46 -05:00
Tim Smith
a81434c74f Removed places where -nobuilduht is added to the command line.
#rb ryan.hummer
#preflight 63c7e95fac35a0e9dad8774d

[CL 23754222 by Tim Smith in ue5-main branch]
2023-01-18 08:01:18 -05:00
bob tellez
9ab9ef6345 #UE Pak performance. Save IoStore response files in the Engine saved folder instead of the Logs folder so they are not considered logs for build systems that look for and process logs.
[CL 23680946 by bob tellez in ue5-main branch]
2023-01-13 12:57:06 -05:00
bob tellez
1e24e30ab3 [Backout] - CL23669350
[FYI] Bob.Tellez
Original CL Desc
-----------------------------------------------------------------
#UE Pak performance. Save IoStore response files in the Engine saved folder instead of the Logs folder so they are not considered logs for build systems that look for and process logs.

[CL 23671673 by bob tellez in ue5-main branch]
2023-01-12 18:36:19 -05:00
bob tellez
a920d38b88 #UE Pak performance. Save IoStore response files in the Engine saved folder instead of the Logs folder so they are not considered logs for build systems that look for and process logs.
[CL 23670619 by bob tellez in ue5-main branch]
2023-01-12 17:29:21 -05:00
gary yuan
b4c229eeb8 Fix ProjectName override bug in OpenEditor build command
Running ParseParamValue with a preset project path adds extra quotes to the output and breaks the IsPathRooted check

#rb eric.knapik, tim.kennedy
#preflight skip

[CL 23631773 by gary yuan in ue5-main branch]
2023-01-10 15:20:50 -05:00
andrew grant
a3993df7e7 BenchmarkBuild
- Fix profiling of -xge and -noxge in the same pass resulting in redundant compile steps (UE-173565)
- Make -noclean also imply -nopostclean
- Add support for -ubtargs / -ubt2args / -ubt3args etc to allow profiling of different UBT args (similar to -cookargs / -cook2args etc)

Ex -

RunUAT BenchmarkBuild -project=Unreal -client -compile -ubt2args="-nodebuginfo" -ubt3args="-nopch" -ubt4args="-disableunity" -noxge

Task Unreal Build Client Win64: 00:08:12
Task Unreal Build Client Win64 (-nodebuginfo): 00:07:41
Task Unreal Build Client Win64 (-nopch): 00:13:43
Task Unreal Build Client Win64 (-disableunity): 00:22:49


#jira UE-173565
#preflight none
#rb swarm
[REVIEW] [at]eric.knapik [at]henrik.karlsson

[CL 23616624 by andrew grant in ue5-main branch]
2023-01-09 13:22:13 -05:00
sergio gardeazabal
c1ec9b119c Adding a way to force compression during staging
#rb josh.adams , justin.marcus
#preflight 63b863ff68068a8bd6353273

[CL 23609354 by sergio gardeazabal in ue5-main branch]
2023-01-08 18:12:06 -05:00
gary yuan
510e3345d1 Make OpenEditor BuildCommand accept rooted overrides for UnrealEditorApp and Project paths instead of trying to find them using Host or Project utils.
#rb eric.knapik, tim.kennedy
#preflight 63b776e0763c6c1064d7fc65

[CL 23600847 by gary yuan in ue5-main branch]
2023-01-06 14:00:02 -05:00
Josh Adams
8aed66dd0c - Added MultiShelve UAT script that can shelve multiple changelists into a single shelf
- Added a Reopen command to P4Utils.cs
- Refactored Edit and Reopen (that take file list) into a BatchCommand function to split up huge file lists across multiple commands to respect a max commandline length
- Increased max comandline length from 1024 to the ProcessStartInfo max of 32699
#rb ryan.hummer,matt.peters
#preflight 63b59866d067708b9bfe3953

[CL 23575766 by Josh Adams in ue5-main branch]
2023-01-04 10:23:56 -05:00