Commit Graph

27 Commits

Author SHA1 Message Date
zousar shaker
e330e9d4c9 Target Iteration: Usability improvements to StorageServerConnection/File:
- Set a lower timeout on the duration of initial handshake (5.0 seconds) so that we don't stall for a long time trying to connect to an invalid host
- Provide a descriptive error messagebox when not in unattended mode and we fail to connect to any host
- Avoid attempting to connect to localhost on non-desktop platforms (they can never be the host, and if we're using IPv6 localhost, it can cause them to try an unnecessary name lookup).

#rb dan.engelbrecht

[CL 27326709 by zousar shaker in ue5-main branch]
2023-08-23 19:13:18 -04:00
dan engelbrecht
06d88ac15e Use the already loaded bytes available even if read overlaps boundary of buffer in FStorageServerFileHandle before
#rb pj.kack per.larsson

[CL 26705396 by dan engelbrecht in ue5-main branch]
2023-07-31 10:12:18 -04:00
zousar shaker
c5d05e1f4e 2nd attempt:
Rework zenserver host address detection when using COTF
Make sure .projectstore file is properly staged in all scenarios

#rb zousar.shaker

[CL 26123110 by zousar shaker in ue5-main branch]
2023-06-20 10:53:32 -04:00
zousar shaker
71aecc1264 [Backout] - CL26049351
[FYI] dan.engelbrecht
Original CL Desc
-----------------------------------------------------------------
Rework zenserver host address detection when using COTF
Make sure .projectstore file is properly staged in all scenarios

#rb zousar.shaker

[CL 26085241 by zousar shaker in ue5-main branch]
2023-06-19 02:23:59 -04:00
dan engelbrecht
9679996964 Rework zenserver host address detection when using COTF
Make sure .projectstore file is properly staged in all scenarios

#rb zousar.shaker

[CL 26049362 by dan engelbrecht in ue5-main branch]
2023-06-16 09:32:36 -04:00
dan engelbrecht
20799c9f48 Store cooker computer remote ips in .projectstore
#rb zousar.shaker

[CL 26006400 by dan engelbrecht in ue5-main branch]
2023-06-15 04:51:13 -04:00
dan engelbrecht
ea0aa7d6b5 Simplify detection of .projectstore file by using the relative content root
#rb pj.kack
#preflight 646e26bf64351d76f3a675ea

[CL 25617036 by dan engelbrecht in ue5-main branch]
2023-05-25 03:42:09 -04:00
dan engelbrecht
96500c25d1 Automatically detect a if a staged target is using zenstore without pak and configure zenserver host and project id.
#jira UE-144425
#rb zousar.shaker pj.kack
#preflight 64427d5b4944ef78923c85b6

[CL 25144415 by dan engelbrecht in ue5-main branch]
2023-04-21 08:56:22 -04:00
dan engelbrecht
db5e568754 Game client no longer needs -zenstoreproject or -zenstorehost arguments to correctly run with a COTF server which uses Zen.
Game client now accepts only -filehostip for connection to a CTOF server, -cookonthefly is no longer necessary and is ignored.
Game client will now receive ProjectName (ProjectId), Platform (OplogId), Zen server host name and port from COTF server if running with zenstore.
Fixed issues with generating the ProjectId from a path where drive letters where lower case.

#rb zousar.shaker pj.kack
#preflight

[CL 24889513 by dan engelbrecht in ue5-main branch]
2023-04-03 01:51:40 -04:00
CarlMagnus Nordin
3d5a2052dc AsyncLoading2: Add support for having multiple PackageStore backends
#preflight 6296fc872a1851b4ccae7e39
#rb pj.kack

[CL 20447121 by CarlMagnus Nordin in ue5-main branch]
2022-06-01 02:12:33 -04:00
CarlMagnus Nordin
0282a5aef6 Unified Zen and legacy COTF network protocols.
*Reduces code duplication
*Enables the same ODSC flow to be used for both COTF variants
*The client will now autodetect if it should run in Zen mode or not
#rb pj.kack,per.larsson
#preflight 628c79bdf057b981ca479b3e

[CL 20344832 by CarlMagnus Nordin in ue5-main branch]
2022-05-24 02:50:39 -04:00
PJ Kack
e9d2d7c1aa ZenCOTF: Fix ODSC related shutdown crashes and deadlocks caused by issueing shader compilation requests after cotf server connection has been closed by StorageServerPlatformFile.
#jira none
#rb carlmagnus.nordin, jason.nadro
#rnx
#preflight 624c49a4cc0872e96b05f10e

[CL 19643400 by PJ Kack in ue5-main branch]
2022-04-06 02:31:41 -04:00
CarlMagnus Nordin
c8943034df ZenCOTF: Fixed shutdown crash caused by server connection outliving the socket system
#rnx
#preflight 621e212fad75a7f58fbf6b33
#rb pj.kack

[CL 19219059 by CarlMagnus Nordin in ue5-main branch]
2022-03-02 02:53:54 -05:00
CarlMagnus Nordin
982d0dc2c3 ZenCOTF: Fixed reversed order between connecting to the cotf server and zen
#rnx
#preflight 61fd4f3d295dd9c101c1a610
#rb pj.kack

[CL 18883399 by CarlMagnus Nordin in ue5-main branch]
2022-02-07 02:47:27 -05:00
CarlMagnus Nordin
9b83a6ffea Moved FCookOnTheFlyPackageStore declaration to header file
#rnx
#rb trivial
#preflight 61fd4a1b51378325c33f4c3d

[CL 18864090 by CarlMagnus Nordin in ue5-main branch]
2022-02-04 11:03:31 -05:00
Martin Ridgers
80a63f44c6 Check local cooked for files that are to be read externally
Currently there are some instances where data must exist on the local file system in order for it to be read by a third party API (Mac's Metal shader libraries being one such example). This change overlays files in Saved/Cooked and checks in there for any external-read path requested that is and prefixed with ../../../
#rb perl
#rnx
#preflight 61fbc8f23b22793ca35698a2

[CL 18841660 by Martin Ridgers in ue5-main branch]
2022-02-03 07:25:17 -05:00
carlmagnus nordin
b49e6d13e2 Made Zen COTF API more private
#rnx
#rb pj.kack
#preflight 61f3c7e0da54035207f6c79f

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18770247 in //UE5/Release-5.0/... via CL 18770252 via CL 18770332
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18770334 by carlmagnus nordin in ue5-main branch]
2022-01-28 08:30:42 -05:00
per larsson
d4d3f66241 Zen - fix bug with missing offset argument
#rb CarlMagnus.Nordin
#rnx
#jira none
#preflight none

#ROBOMERGE-AUTHOR: per.larsson
#ROBOMERGE-SOURCE: CL 18615694 in //UE5/Release-5.0/... via CL 18615702 via CL 18615712
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)

[CL 18615718 by per larsson in ue5-main branch]
2022-01-14 05:00:42 -05:00
zousar shaker
0832797d28 Ensure that Zen ProjectIds are unique for projects with the same name in differing roots (or in the same root) while attempting to maintain some easily recognizable desriptors for the project.
Staged builds are explicitly assigned a project id via the persistent UECommandline.txt file.  This works on mobile/consoles, but isn't implemented for Windows/Linux/Mac, so staged builds for those platforms don't have a persistent assignment of project id and instead rely on the launch by the editor including the appropriate commandlines.
Unstaged builds on Windows/Mac/Linux now determine project id independently after they have determined the project file path.  This happens after the project file has been determined and has had case corrected.  This ensures it is different/unique for multiple blueprint projects that share the same executable.
Fix a bug where second cook in editor to zen would stall because the queue thinks all additions are finished due to a boolean that isn't reset.

#rb devin.doucette
#rb per.larsson
#preflight 61b39a35a2562c8b1c40f81e

#ROBOMERGE-AUTHOR: zousar.shaker
#ROBOMERGE-SOURCE: CL 18432317 in //UE5/Release-5.0/... via CL 18435404
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)

[CL 18435895 by zousar shaker in ue5-release-engine-test branch]
2021-12-10 18:06:39 -05:00
aurel cordonnier
a6e741e007 Merge from Release-Engine-Staging @ 17915896 to Release-Engine-Test
This represents UE4/Main @17911760, Release-5.0 @17915875 and Dev-PerfTest @17914035

[CL 17918595 by aurel cordonnier in ue5-release-engine-test branch]
2021-10-25 20:05:28 -04:00
carlmagnus nordin
1a96030cec Added missing include
#rnx

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17643126 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v875-17642767)

[CL 17643129 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-28 05:00:12 -04:00
carlmagnus nordin
51c9794dea PackageStore refactor
* Decouple container concept from IoDispatcher
* Decoiuple PackageStore implementation from AsyncLoading2
* Restore ucas unmount fix that got kist when merrging from UE4
* Fix packages being left in the PackageStiore even after unmounting contaiiners

#rnx
#rb pj.kack, per.larsson
#preflight 61520cc52afc2d0001146ce7

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 17641845 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v874-17637634)

[CL 17642353 by carlmagnus nordin in ue5-release-engine-test branch]
2021-09-28 04:00:33 -04:00
devin doucette
b611884c48 IoDispatcher: Always initialize IoDispatcher
Only return true from FIoDispatcher::IsInitialized() when the script objects chunk exists to keep its behavior consistent. Later, this function will be deprecated in favor of a function on IAsyncPackageLoader.

#rb Zousar.Shaker
[FYI] CarlMagnus.Nordin
#preflight 614541e1bf494a0001bd76bc

#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17570905 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v871-17566257)

[CL 17570916 by devin doucette in ue5-release-engine-test branch]
2021-09-20 11:00:30 -04:00
per larsson
65502f9636 Zen: Changed file chunk ID to be based on filename hash instead of a monotonically increasing index
#rb CarlMagnus.Nordin
#rnx
#jira none

#ROBOMERGE-SOURCE: CL 17230621 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)

[CL 17230626 by per larsson in ue5-release-engine-test branch]
2021-08-19 06:42:05 -04:00
per larsson
e29cc6c467 Simplify command line arguments when running COTF with Zen loader/storage server
- changed Cooker arguments from -CookOnTheFly -IoStore to -CookOnTheFly -ZenStore
- changed Zen host argument from -StorageServerHost to -ZenStoreHost
- updated UAT to support running CBTB with -ZenStore
- updated StorageServerConnection to handle multiple hosts from command line

Cooker:
  CBTB: -ZenStore
  COTF: -ZenStore -CookOnTheFly

Game:
  CBTB: -ZenStoreHost=<ip> or <ip1,ip2>
  COTF: -ZenStoreHost=<ip> or <ip1,ip2> -CookOnTheFly

#rb CarlMagnus.Nordin
#jira none
#rnx

#ROBOMERGE-SOURCE: CL 16849969 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16849982 by per larsson in ue5-release-engine-test branch]
2021-07-14 10:40:16 -04:00