Commit Graph

1116 Commits

Author SHA1 Message Date
Ben Marsh
cc7c17c23f Horde: Additional logging for running of compute commands.
#preflight none

[CL 24542938 by Ben Marsh in ue5-main branch]
2023-03-07 12:39:50 -05:00
Ben Marsh
bcd78e2093 Horde: Fix decoding of compute errors.
#preflight none

[CL 24542823 by Ben Marsh in ue5-main branch]
2023-03-07 12:33:21 -05:00
Ben Marsh
2036e3f2a6 Horde: Add an error return path for C++ compute messages.
#preflight none

[CL 24539189 by Ben Marsh in ue5-main branch]
2023-03-07 09:04:40 -05:00
Tim Smith
62308e9b03 Fix coding standard issues in some RigVM code.
#rb self
#rnx
#preflight 64074165747339025abcd957

[CL 24539174 by Tim Smith in ue5-main branch]
2023-03-07 09:03:07 -05:00
Helge Mathee
3e9e4bf9d7 UHT: Fix typo in
#preflight https://horde.devtools.epicgames.com/job/640730d7ba12ba641680cb70
#jira na
#fyi tim.smith
#fyi jaime.cifuentes

[CL 24538574 by Helge Mathee in ue5-main branch]
2023-03-07 08:02:09 -05:00
Ben Marsh
88ef609790 Horde: Fix rare data corruption when streaming data into bundles.
If data is being appended to an existing file node in chunks, and the bundle size exceeds the desired max while data is still in the buffer, a new bundle (and write buffer) will be created without preserving the partially written data. Resolution is to only flush bundles when complete nodes have been added.

#preflight none

[CL 24532143 by Ben Marsh in ue5-main branch]
2023-03-06 17:24:40 -05:00
Ben Marsh
320107efd8 Horde: Compute service refactor.
- Message framing changed back to use simple
- Support for multiplexing multiple requests over a single socket. Requests are tagged with an arbitrary channel id and routed to a queue for the appropriate IComputeChannel on the client.
- Clients can now request a message writer through the IComputeChannel interface, allowing pooling of write buffers for generating new messages and reducing data copies.
- Added accounting for machine resources in compute leases. Compute requests specify min/max desired resources, receive allocated resources in respoonse.
- Switched to AES-GCM for encryption. Message length is unencrypted but validated as part of the message packet.

#preflight none

[CL 24521844 by Ben Marsh in ue5-main branch]
2023-03-06 09:55:49 -05:00
Ben Marsh
f69c510598 EpicGames.Serialization: Fix writing of CB fields at the root of the output object.
* CbWriter was expecting BeginObject/BeginArray call before any fields were written, so did not handle writing fields directly to the root scope.
* Fields written at the root scope were not being flushed immediately after writing. Rest of the class assumes that any complete fields at root scope had already been accounted for in object size.

#preflight none
#fyi Joakim.Lindqvist

[CL 24520927 by Ben Marsh in ue5-main branch]
2023-03-06 09:00:40 -05:00
Joakim Lindqvist
7df0ddc0de OidcToken - Ignore stored tokens if a "Key not valid for use in specified state" exception is raised. Fixes rare issue issue on Windows machiens were the stored token can not be read.
#jira UE-178675
#preflight none

[CL 24519691 by Joakim Lindqvist in ue5-main branch]
2023-03-06 05:20:27 -05:00
Ben Marsh
6fac200317 EpicGames.Serialization: Refactoring CbWriter.
* New base class (CbWriterBase) has a virtual method to allocate chunks of memory for output, while still sharing common functionality for tracking scope state.
* Data for fixing up size/offset fields is written to blocks of memory returned by the CbWriterBase allocator rather than just to the final output buffer.
* Added support for embedded externally allocated blocks of data into the output stream, correctly updating size fields of containing fields.

#preflight none
#fyi Joakim.Lindqvist

[CL 24505997 by Ben Marsh in ue5-main branch]
2023-03-03 15:37:45 -05:00
Ben Marsh
83742f4f1b Horde: Allow specifying a range of required resources when requesting a compute lease, and return the allocated resources to the requestor.
#preflight none

[CL 24505811 by Ben Marsh in ue5-main branch]
2023-03-03 15:28:01 -05:00
dmytro vovk
4a7ecdab44 UObject data layout optimization
#rb Steve.Robb Tim.Smith
#preflight 64009fc2c63aaf1e135a408c

[CL 24482248 by dmytro vovk in ue5-main branch]
2023-03-02 10:35:18 -05:00
Ben Marsh
4b6684786b Horde: Change compute client to connect directly to remotes rather than opening a listen socket.
#preflight none

[CL 24474809 by Ben Marsh in ue5-main branch]
2023-03-01 19:51:33 -05:00
tim smith
8d9991b0ae Remove the comment and tooltip metadata from shipping builds.
#rb rex.hill
#preflight 63ffa785df66ed5fc10c2066

[CL 24471988 by tim smith in ue5-main branch]
2023-03-01 17:15:38 -05:00
Joe Kirchoff
1b044d8f36 EpicGames.Build: Don't throw an exception if a path in AdditionalSearchPath doesn't exist
#rnx
#rb trivial
#preflight 63ff8a2ff43e53f68137ead4

[CL 24465656 by Joe Kirchoff in ue5-main branch]
2023-03-01 12:46:25 -05:00
Ben Marsh
d55712d7de EpicGames.Core: Fix warnings generating documentation.
#preflight none

[CL 24457120 by Ben Marsh in ue5-main branch]
2023-02-28 21:58:32 -05:00
Ben Marsh
101ffd3de1 Horde: Connect directly to agents when running remote execution leases, rather than having agents connect back to the initiator.
- Agents now accept incoming connections on a defined port, and match them up with nonces specified through leases assigned from the server.
- Requests for new compute resources now attempt to allocate a lease and return immediately with nonce and AES keys.

#preflight none

[CL 24452056 by Ben Marsh in ue5-main branch]
2023-02-28 16:43:25 -05:00
Ben Marsh
a1f6349f41 Horde: Add Json converter for StringId.
#preflight none

[CL 24432005 by Ben Marsh in ue5-main branch]
2023-02-27 16:16:55 -05:00
Tim Smith
71de75cc14 Reduced the lock contention while updating the queue empty event.
#rb dan.engelbrecht
#rnx
#preflight 63fcb415ef1b24bf94937521

[CL 24422210 by Tim Smith in ue5-main branch]
2023-02-27 08:55:38 -05:00
Ben Marsh
b6f9841927 EpicGames.Core: Add missing equality and comparison operators causing static analysis warnings.
#preflight none

[CL 24411440 by Ben Marsh in ue5-main branch]
2023-02-24 18:12:33 -05:00
Ben Marsh
dcda73f48a Horde: Add a service to allow tunneling connections through the Horde server. Accepts incoming connections on two ports and matches them up via a cryptographic unique id. Designed for use by compute service, where client machines may not accept incoming connections.
#preflight none

[CL 24411094 by Ben Marsh in ue5-main branch]
2023-02-24 17:55:04 -05:00
Ben Marsh
0146eb14d0 EpicGames.Core: Fix NET 3.1 compilation of ByteString struct.
#preflight none

[CL 24405723 by Ben Marsh in ue5-main branch]
2023-02-24 14:12:54 -05:00
Ben Marsh
164452fbac EpicGames.Core: Add a utility class for manipulating byte strings, allowing value-like comparison for equality and sorting.
#preflight none

[CL 24401948 by Ben Marsh in ue5-main branch]
2023-02-24 11:24:11 -05:00
Ben Marsh
bc5f5efa44 UGS: Fix threading issue when opening multiple startup projects with OIDC configuration.
#preflight none

[CL 24401761 by Ben Marsh in ue5-main branch]
2023-02-24 11:15:22 -05:00
Ben Marsh
6915c52388 Horde: C++ compute improvements.
* Add an in-memory cache for data retrieved from the initiator.
* Allow appending partial line chunks to LogChunkNode instances.
* Allow constructing a server compute handler instance with the URL of a server rather than a HttpClient factory.

#preflight none

[CL 24386555 by Ben Marsh in ue5-main branch]
2023-02-23 14:40:58 -05:00