Commit Graph

1296 Commits

Author SHA1 Message Date
zousar shaker
8b0a81dc6c Add NoZenLocalFallback and InstalledNoZenLocalFallback DDC graphs for use in scenarios where users are unable to run a local zenserver for any reason. Also fix a mis-labeling of a local cache store as being remote if it is deactivated.
#rb devin.doucette
#jira UE-208878

[CL 32022101 by zousar shaker in ue5-main branch]
2024-03-05 11:24:22 -05:00
paul chipchase
1f773a8c4c Split engine.ini:[Ias]:ForceTocFromMountedPaks into two different values.
#rb Martin.Ridgers
#jira UE-189912
#rnx

- When it was added ForceTocFromMountedPaks controlled if we should create .uondemandtoc files when building IAS data and if we should use them at runtime, this was inflexiable for testing purposes so has been split into:
-- engine.ini:[Ias]:CreatePerContainerTocs which controls if we should create the files.
-- engine.ini:[Ias]:UsePerContainerTocs which controls if we should run from the files.
- Both values remain defaulted to off so that the system is only used as opt in.

[CL 31943526 by paul chipchase in ue5-main branch]
2024-03-01 10:14:45 -05:00
kevin macaulayvacher
abfebd7d6f Improves the performance of CoreRedirects
::Before::
- Boot -> PIE (6m3s)
- FCoreRedirects::RedirectNameAndValues: 18s

::After::
- Boot -> PIE (5m41s)
- FCoreRedirects::RedirectNameAndValues: 5.3s


- MatchWildcard=true support has been added to how CoreRedirects are defined in INI files. You may now define OldName="...SomeSuffix", OldName="SomePrefix..." or OldName="...SomeSubstring..." to perform string manipulation based matching for redirects, deprecating MatchSubstring=true
- Substring matches are very expensive compared to pure FName matching or even prefix and suffix matching. This expense is problematic since the presence of a substring matching rule can mean that all queries for a particular redirect type, such as package name, will need to perform the substring check even if a substring match is relatively uncommon. To combat this cost, we add a reimplementation of the PM-k, (PredictMatch-8) algorithm. This allows a redirect query to check for substring matches against _all_ possible substrings with a single pass over the redirect query name. The fuzzy matching is designed such that false positives may occur but not false negatives. When a fuzzy match is found we must perform the slow substring match to confirm. However fuzzy matching allows use to dramatically reduce our string scannin. In a case where there were 4 MatchSubstrings, from boot to PIE, we now perform slow substring matches < 0.8% of the time compared to previously.
- Replaces `FString` manipulation during redirect queries to instead use a small wrapper type `FNameUtf8String` which provides a small API to allow for FName comparisons, building and ultimately reducing copies. UTF8 is used since we need to ensure we have an encoding that we can keep the same for storing wildcard strings that will match how we query for wildcard matches. UTF8 is byte efficient for ANSI text, which improves fuzzy match prediction.

==========================================
Substring Stats
Substring Lookups:                24342736
Prediction Hit:                   72
Prediction Miss:                  213377
==========================================

#jira UE-204063
#rb Matt.Peters
[FYI] Francis.Hurteau

[CL 31941627 by kevin macaulayvacher in ue5-main branch]
2024-03-01 08:57:14 -05:00
mark lintott
e66322fd4d Studio Telemetry Configuration Changes
Moved the provider cofiguration settings to the StudioTelemetry.Provider.ProviderName section
Added StudioTelemetry global configuration settings to the StudioTelemetry.Config section
[FYI] Wes.Hunt
#rb Wes.Hunt

[CL 31867515 by mark lintott in ue5-main branch]
2024-02-28 05:48:05 -05:00
jeremie roy
00d2959286 AppleARKit - By default, prevent ARKit from starting its listener.
- You can override this to revert to the previous behavior by specifying -EnableLiveLinkFaceTracking on the command line.
- Modify ARKit module to not bind to its port by default unless specified by the command line
- Move ARKit server binding to its source instead of the module

This fixes an issue where starting two editors would make it impossible for one of them to use the ARKit livelink plugin since only one could bind to the ARKit port.
Moving it to the livelink source allows the user to pick which UE instance should bind the port.

#jira UE-205785
#rb jason.walter

[CL 31820784 by jeremie roy in ue5-main branch]
2024-02-26 18:09:46 -05:00
lorry li
3019b9090b Adjust the max concurrent http requests, make it hotfixable.
[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]ryan.hairyes [at]chris.varnsverry [at]laura.hermanns [at]alejandro.aguilar
#rb Chris.Varnsverry, Laura.Hermanns, michael.atchison

[CL 31817974 by lorry li in ue5-main branch]
2024-02-26 16:50:00 -05:00
lorry li
6401069a66 Change the default connection timeout to 30s.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]eric.day [at]rafa.lecina
#rb eric.day, michael.atchison, Rafa.Lecina

[CL 31774242 by lorry li in ue5-main branch]
2024-02-23 15:17:48 -05:00
lorry li
e4db25bf24 Get rid of the legacy http flush in websockets when suspend.
[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]rafa.lecina [at]evgenii.babinets
#rb michael.atchison

[CL 31700324 by lorry li in ue5-main branch]
2024-02-21 17:41:38 -05:00
lorry li
76603dbcc8 Revert changelist which changed the concurrent http requests limit on release branch, which is not ready yet.
[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]alejandro.aguilar [at]jan.orlowski [at]stephen.ma

#rb alejandro.aguilar, michael.atchison, Michael.Kirzinger, stephen.ma

[CL 31685920 by lorry li in ue5-main branch]
2024-02-21 11:42:26 -05:00
carl lloyd
75eada5e30 Disable heaps on AMD\Intel platforms due to lack of shared memory support and heaps being a requirement only for Apple Sillicon
#rb Zack.Neyland
#jira UE-207397
#lockdown jessica.agee

[CL 31667719 by carl lloyd in ue5-main branch]
2024-02-20 20:31:08 -05:00
zousar shaker
a926360c9d Change default zen data and install directories to be per-user, and clean up old default directories when migrating to a new one.
#rb dan.engelbrecht
#jira UE-205982

[CL 31567090 by zousar shaker in ue5-main branch]
2024-02-16 11:15:16 -05:00
carl lloyd
ba27035a00 Added support for Metal Shader Converter with Bindless in MetalRHI
- Available on SM6
 - Disabled with config
 - Includes support for heap allocations on Metal, enabled by default, can be disabled with -nometalheap

#rb Luke.Thatcher, Chris.Waters, Laura.Hermanns
#jira UE-204112

[CL 31531113 by carl lloyd in ue5-main branch]
2024-02-15 14:58:02 -05:00
florin pascu
b10df2c9a1 Re-submit Desktop Platforms TP Refactor TPC/TPS Windows/Mac/Linux/WINGDK
Section Names for Windows/Mac/Linux/WindGDK will use the old names for compatibility.
#jira UE-206488
#rb Brandon.Schaefer, Josh.Adams

[CL 31522656 by florin pascu in ue5-main branch]
2024-02-15 11:46:19 -05:00
matt peters
9c57b0255d AssetRegistry: Disable Discovery cache and use of PlatformFileJournal. It needs to be fixed to work with symbolic links and with some Windows11 edge cases.
#jira UE-206631
#jira UE-203531
#rnx
#rb danny.couture
#rbself trivial, notified users

[CL 31423165 by matt peters in ue5-main branch]
2024-02-13 09:04:14 -05:00
alejandro aguilar
bec1ce5f25 Moved HTTP Threaded Request limit out of general config and into PS5 specific config. Updated rate to the maximum without errors after testing.
[REVIEW] [at]Chris.Varnsverry
#rb Chris.Varnsverry

[CL 31287814 by alejandro aguilar in ue5-main branch]
2024-02-08 07:22:58 -05:00
chris babcock
8e3a5377b0 Add option to request microphone permission for AndroidVoice module at startup (SplashActivity)
#jira UE-187580
#android
[REVIEW] [at]Ben.Temple
#rb Ben.Temple

[CL 31243927 by chris babcock in ue5-main branch]
2024-02-06 20:47:30 -05:00
mark lintott
b3e71c3191 Updated ini file to include an example Horde Analytics setup
[FYI] patrick.finegan

[CL 31168168 by mark lintott in ue5-main branch]
2024-02-05 05:12:03 -05:00
calvin zheng
26de99eff3 UBT handling Privacy Manifest files for iOS and Mac
#jira UE-191438
#rb adam.kinge, zack.neyland

[CL 31138198 by calvin zheng in ue5-main branch]
2024-02-02 15:34:12 -05:00
louisphilippe seguin
e9a1289d4e Iris
* Rename Prioritizer configs to make them easier to search for

#rb Peter.Engstrom

[CL 31125463 by louisphilippe seguin in ue5-main branch]
2024-02-02 10:32:46 -05:00
rafa lecina
d0bea81646 Fix HttpActivityTimeout to be lower than HttpConnectionTimeout
#jira  UE-205423
[REVIEW] [at]Lorry.Li

#rb lorry.li

[CL 31093528 by rafa lecina in ue5-main branch]
2024-02-01 13:06:39 -05:00
joakim lindqvist
6f698f031d Fixed incorrect comment, HttpVersion applies to HttpBackends not FilesystemBackends in DDC.
[CL 31001134 by joakim lindqvist in ue5-main branch]
2024-01-30 07:11:42 -05:00
max chen
cb7cce86db Sequencer: Move SequenceTimeUnit to MovieScene module and rename as MovieSceneTimeUnit
Added redirector
Tested that a SequenceTimeUnit is automatically upgraded to a MovieSceneTimeUnit

#rb ludovic.chabant

[CL 30931619 by max chen in ue5-main branch]
2024-01-26 13:22:30 -05:00
paul chipchase
7ada78893d Add config file options so that the virtualization system is able to retry pulling failed payloads when running in unattended mode (defaults to off)
#rb PJ.Kack
#jira UE-203381

-Some users have reported seeing their long cooks fail due to a machine having an unreliable connection or networkcard. In these cases the network outage can be swiftly fixed but if VA failed to pull a payload before then, the cook will terminate and need to be restarted which can cost a lot of time.
- It was requested that we add an optional way to have the system retry payload pulling when running in unattended mode but also to wait for X time (usually many minutes) before trying again. If it is likely that the connection will be restored within those few minutes then waiting will be much less costly than restarting a cook.

- That payloads can be pulled on many threads at the same time makes the logic a little tricky, so rather than counting how many payloads have failed vs the retry counter we count how many times we've logged a message to the user as this logging is protected by a critical section and acts as a way to "group" together failed pulls that occur around about the same time. We then reset this counter to 0 if we detect a successful pull.
- It is possible that a pull fails because the payload is missing, in which case this logic will probably cause the counter to reset frequently and the error to not become fatal for quite some time (possibly until the cook has almost finished) but it is quite unlikly to occur and due to this I have favored erring towards simple code rather than trying to track individual payload failures vs grouped failures vs successful pulls.

- Note: That when backends fail to pull payloads they generally log errors, which will eventually cause most of our processes to return non zero to indicate failure. VA should not log errors while we are inside of a retry loop and only print out errors when we detect a problem that we cannot solve to avoid this. This is being addressed as it's own work item.

[CL 30925886 by paul chipchase in ue5-main branch]
2024-01-26 11:09:38 -05:00
paul chipchase
79aa0812dd Creating a .uondemandtoc file per container can now be enabled via the config file system. Continues to default to disabled.
#rb Martin.Ridgers
#jira UE-189912
#rnx

- This continues to be a temp measure while this feature is disabled for testing, the only reason for the change is so that the feature can be enabled for testing via data rather than by changing code.
- Enabled by setting engine.ini:[Ias]:ForceTocFromMountedPaks to true, note that setting this for a specific project doesn't work as UnrealPak does not load per project config settings.

[CL 30838003 by paul chipchase in ue5-main branch]
2024-01-24 05:26:22 -05:00
lorry li
0980390e94 Migrate retry system total timeout for http request, by adding total timeout feature in http request;
Deprecate HttpTimeout config, use HttpActivityTimeout or HttpTotalTimeout instead;
Deprecate HttpSendTimeout config, only use HttpActivityTimeout;
Make HttpActivityTimeout work on all platforms, not only CurlHttp;
Added corresponding http tests;
Now because timeout migrated, enable new flow by default in retry system to have non-game thread support.

#jira UE-197485, UE-202201
[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]rafa.lecina
#rb michael.atchison, Michael.Kirzinger
#tests Tested through WebTests project on all platforms, also tried the game on PC.

[CL 30817277 by lorry li in ue5-main branch]
2024-01-23 16:08:08 -05:00