Commit Graph

34 Commits

Author SHA1 Message Date
Ben Marsh
0e4f348199 Horde: Move IStorageClient implementations into the EpicGames.Horde.Storage.Clients namespace.
[CL 27738432 by Ben Marsh in ue5-main branch]
2023-09-09 10:14:12 -04:00
Ben Marsh
4d8cd83a73 Horde: Automatically insert the CLIENT/REMOTE prefix to log messages output by the local compute client, and have the agent channel output use the same logger.
[CL 27248101 by Ben Marsh in ue5-main branch]
2023-08-21 14:38:59 -04:00
carl bystrom
72f86897d5 Horde: Support running compute tasks under Wine on Linux
Compute tasks can request Wine to be used and agent can choose to respect that if conditions are right (running on Linux and the Wine executable path set).

#rb Ben.Marsh

[CL 27029748 by carl bystrom in ue5-main branch]
2023-08-11 11:25:26 -04:00
Ben Marsh
705931a7ec Horde: Unifying C#/C++ implementations of remote execution code.
- Handle ref counting in the same way for each implementation.
- Writers now need to be created explicitly, and are marked complete when their last writer is released.
- Added -InProc command line argument to sample app, which runs C# assemblies in the same process when possible.

[CL 26675350 by Ben Marsh in ue5-main branch]
2023-07-28 11:51:34 -04:00
Ben Marsh
f9e052f6bd Horde: Sync C# memory layout of compute buffer with the C++ version.
[CL 26609902 by Ben Marsh in ue5-main branch]
2023-07-26 12:07:12 -04:00
Ben Marsh
52ecb3a14a Horde: Change IComputeSocket to a concrete base class.
[CL 26508737 by Ben Marsh in ue5-main branch]
2023-07-21 08:49:43 -04:00
Ben Marsh
31a83fc628 Horde: Move bundle implementation into a separate namespace.
[CL 26226699 by Ben Marsh in ue5-main branch]
2023-06-24 16:37:55 -04:00
Ben Marsh
b7f9afb66a Horde: Add a custom handle type for nodes stored in bundles, abstracting implementation details away from most clients.
[CL 26226673 by Ben Marsh in ue5-main branch]
2023-06-24 16:22:48 -04:00
Ben Marsh
7d5e31784a Horde: Start merging implementations of blobs and nodes in IStorageClient.
* Rename NodeType -> BlobType, NodeData -> BlobData, and NodeHandle -> BlobHandle.
* Add methods to BlobHandle to allow partial reads of payload, and querying refs and type without fetching all data.

[CL 25895881 by Ben Marsh in ue5-main branch]
2023-06-09 11:57:14 -04:00
Ben Marsh
3b71eb6c0e Horde: Flush bundle writers before disposing of them. Ensures that node handles are not corrupted due to buffered data existing only in rented memory.
[CL 25864640 by Ben Marsh in ue5-main branch]
2023-06-07 22:20:37 -04:00
Ben Marsh
d9c08c3a83 Horde: Prevent direct access to locators in node handles, to allow implementations to delay computation until they have been flushed.
[CL 25850133 by Ben Marsh in ue5-main branch]
2023-06-07 13:19:54 -04:00
Ben Marsh
c6246d9472 Horde: Reverting CL 25709378; storing hashes with node locators/handles removes a lot of complexity, and makes it easier for clients to cache data.
#preflight 647a06955e71ee4db648158f

[CL 25762879 by Ben Marsh in ue5-main branch]
2023-06-02 11:35:51 -04:00
Ben Marsh
a4e2059326 Horde: Abstract out logic for building bundles into an interface, so implementations can choose not to use it if they desire.
#preflight 6477f5eb313d43282fcf9a9f

[CL 25724155 by Ben Marsh in ue5-main branch]
2023-06-01 08:39:27 -04:00
Ben Marsh
221ce42510 Horde: Remove hashes from node handles by default. A separate HashedNodeHandle type bundles the handle with a hash value for round-tripping data through NodeRef objects.
#preflight none

[CL 25709378 by Ben Marsh in ue5-main branch]
2023-05-31 14:41:23 -04:00
Ben Marsh
d5e215945e Horde: Rename RemoteComputeSocket -> ComputeSocket. Whether it's remote or not is determined by the endpoint parameter.
#preflight none

[CL 25564036 by Ben Marsh in ue5-main branch]
2023-05-22 10:44:26 -04:00
Ben Marsh
600063417c Horde: Remove IAgentMessage interface; expose the concrete type instead.
#preflight none

[CL 25559209 by Ben Marsh in ue5-main branch]
2023-05-21 09:27:26 -04:00
Ben Marsh
4de1cf0f3b Horde: Rename compute classes designed solely for communicating with the Horde agent to distinguish from the general-purpose compute API.
#preflight none

[CL 25558029 by Ben Marsh in ue5-main branch]
2023-05-20 15:36:19 -04:00
Ben Marsh
09a3e5c540 Horde: Remove special control messages for notifying on buffer attachment.
This is trivially handled with explicit messaging from the agent, and can be implemented as necessary for direct channel comms.

#preflight none

[CL 25557872 by Ben Marsh in ue5-main branch]
2023-05-20 14:50:49 -04:00
Ben Marsh
5f6c49fe70 Horde: Refactor compute API to support attachment of arbitrary buffers from workers.
#preflight none

[CL 25555811 by Ben Marsh in ue5-main branch]
2023-05-19 21:50:03 -04:00
Ben Marsh
ac2891dda7 Horde: Remove methods from IComputeSocket which are only implemented by the transport-based implementation. Expose implementations of IComputeSocket to consumers.
#preflight none

[CL 25544993 by Ben Marsh in ue5-main branch]
2023-05-19 11:00:44 -04:00
Ben Marsh
da8149d19b Horde: Various compute API improvements.
* All standard buffers now implement IComputeBuffer directly, and do not need to have ToSharedInstance() called to create a ref-counted version.
* Workers can now construct a socket directly, allowing multiple buffers to be attached.
* Control messages are sent whenever a receive buffer is attached, allowing the remote to wait for it to be available.

#preflight none

[CL 25169238 by Ben Marsh in ue5-main branch]
2023-04-24 15:21:04 -04:00
Ben Marsh
adcdf64794 Horde: Support communicating with child processes via custom channel ids, making it possible to run multiple processes on forked message loops.
#preflight none

[CL 25136094 by Ben Marsh in ue5-main branch]
2023-04-20 16:15:10 -04:00
Ben Marsh
8c65a920e9 Horde: Add support for forking the remote message loop during remote execution.
#preflight none

[CL 25134220 by Ben Marsh in ue5-main branch]
2023-04-20 15:15:34 -04:00
Ben Marsh
f9262c451a Horde: Trigger a new OIDC login if the current refresh token has expired.
#preflight none

[CL 24997745 by Ben Marsh in ue5-main branch]
2023-04-11 16:41:59 -04:00
Ben Marsh
13820bd279 Horde: More resilient handling of graceful and forced shutdown scenarios.
* Graceful termination now shuts down the write end of the underlying socket, causing the remote to do the same first. We can then wait for background send/recv tasks to complete normally rather than requiring cancellation.
* IComputeBuffer now functions as a single handle to a shared reference-counted implementation, allowing the socket to acquire its own reference to the buffer and release it once done.

#preflight none

[CL 24995838 by Ben Marsh in ue5-main branch]
2023-04-11 15:00:24 -04:00