Commit Graph

1176 Commits

Author SHA1 Message Date
Ben Marsh
82015f956f Horde: Remote execution improvements.
* Make it an error to send data to a buffer that is not yet attached.
* Send a 'ready' message whenever the remote attaches a message buffer, allowing synchronizing the two machines before streaming data.
* Add a class for tracking remote processes, and reading output data line-by-line.

#preflight none

[CL 24828683 by Ben Marsh in ue5-main branch]
2023-03-28 18:22:19 -04:00
bryan sefcik
f541d9983e Updated finding rule files to be thread safe.
#fyi joe.kirchoff
#preflight 64233839b72410fc17e9aa40

[CL 24824452 by bryan sefcik in ue5-main branch]
2023-03-28 15:24:54 -04:00
Ben Marsh
7338989c6f Horde: Fix compute exceptions being sent with optional descriptions.
#preflight none

[CL 24824131 by Ben Marsh in ue5-main branch]
2023-03-28 15:11:55 -04:00
Ben Marsh
71df3dc928 Horde: Improve cancellation handling for compute workers. Also add default cancellation token parameters to various methods.
#preflight none

[CL 24823233 by Ben Marsh in ue5-main branch]
2023-03-28 14:20:17 -04:00
Ben Marsh
235bdf71c3 Horde: Support cancellation of the child process during remote execution.
#preflight none

[CL 24822394 by Ben Marsh in ue5-main branch]
2023-03-28 13:45:09 -04:00
Ben Marsh
9b0978b04c Horde: Additional logging for spawning child processes.
#preflight none

[CL 24820928 by Ben Marsh in ue5-main branch]
2023-03-28 12:24:35 -04:00
Ben Marsh
21a289b71f Horde: Add a test which creates a bundle manually and using TreeWriter, and checks them for equality.
#preflight none

[CL 24817463 by Ben Marsh in ue5-main branch]
2023-03-28 09:09:07 -04:00
Ben Marsh
a1c859c04c Horde: Fix files not being truncated when extracted from a bundle, resulting in files being longer than they should be.
#preflight none

[CL 24813304 by Ben Marsh in ue5-main branch]
2023-03-27 20:06:39 -04:00
Steve Robb
d87d4e444e Fixed the unsized properties which have been broken since CL# 3494741 by assuming that int and unsigned int are always 32-bit.
#rb tim.smith, devin.doucette
#jira none
#preflight 6421d4f6c42c7fd1cb953d5c

[CL 24805833 by Steve Robb in ue5-main branch]
2023-03-27 13:49:14 -04:00
Ben Marsh
bc2e67af6e Horde: Move the worker message loop into EpicGames.Horde, making it easier to test locally using the LocalComputeClient.
#preflight none

[CL 24805341 by Ben Marsh in ue5-main branch]
2023-03-27 13:27:23 -04:00
Ben Marsh
45c0751119 Horde: Missing file.
#preflight none

[CL 24804939 by Ben Marsh in ue5-main branch]
2023-03-27 13:11:43 -04:00
Ben Marsh
902b731301 Horde: Allow message buffers to automatically flush if there is insufficient space remaining.
* CreateMessage is now CreateMessageAsync, and blocks on a flush if it needs more space.
* Decreased default buffer sizes for compute handler.

#preflight none

[CL 24804925 by Ben Marsh in ue5-main branch]
2023-03-27 13:11:15 -04:00
Ben Marsh
114c44cc15 Horde: Allow compute buffers to be reset back to empty once the reader catches up to the writer.
#preflight none

[CL 24796791 by Ben Marsh in ue5-main branch]
2023-03-25 15:16:15 -04:00
Ben Marsh
5b326d9d03 UGS: Fix parsing of datetime strings with a period in the timezone name (eg. "W. Europe Standard Time").
#preflight none

[CL 24790267 by Ben Marsh in ue5-main branch]
2023-03-24 19:06:13 -04:00
Joe Kirchoff
2a261c113d EpicGames.Core: Obsolete ReadOnlyHashSet in favor of IReadOnlySet
#rb Ben.Marsh
#preflight 641e1145a86ae7cbcc86bc53

[CL 24789316 by Joe Kirchoff in ue5-main branch]
2023-03-24 17:59:47 -04:00
bryan sefcik
3eaa022618 Updated several places in UBT to be thread safe.
#fyi joe.kirchoff
#preflight 641de24b7703a8de3ddb63a7

[CL 24785255 by bryan sefcik in ue5-main branch]
2023-03-24 15:42:24 -04:00
Ben Marsh
fdcbbf8d43 Horde: Add an exception when there is insufficient space in the compute buffer to write a message of the requested size.
#preflight none

[CL 24782342 by Ben Marsh in ue5-main branch]
2023-03-24 13:20:05 -04:00
Ben Marsh
32b9b8309a Horde: Build fix.
#preflight none

[CL 24782270 by Ben Marsh in ue5-main branch]
2023-03-24 13:13:57 -04:00
Ben Marsh
783d78f382 Horde: Return leases directly from compute clients, rather than invoking a callback with the allocated lease.
#preflight none

[CL 24781278 by Ben Marsh in ue5-main branch]
2023-03-24 12:41:04 -04:00
Ben Marsh
bde614a474 Horde: Missing file.
#preflight none

[CL 24779529 by Ben Marsh in ue5-main branch]
2023-03-24 10:56:55 -04:00
Ben Marsh
ddae4304bd Horde: Various remote execution improvements.
* Add a sample app which uploads an application, streams values to it, and prints the output it echoes back to us.
* Add an in-memory transport which uses a C# pipeline to buffer data.
* Add a client implementation that launches a local Horde agent and connects directly to it.
* Add an IPC message channel to allow attaching new shared memory buffers from remote applications.
* Simplify the API for establishing connections.

#preflight none

[CL 24779164 by Ben Marsh in ue5-main branch]
2023-03-24 10:32:39 -04:00
tim smith
27d570f1b3 Minor performance improvement in C# UHT file reading by limiting the buffer size to reduce GC stress.
#rb self
#preflight 641b412a32723d4a202d298d

[CL 24756434 by tim smith in ue5-main branch]
2023-03-22 19:44:33 -04:00
Ben Marsh
bcfbecd121 Horde: Add another method of running compute connections locally, which spawns a separate process without server involvement and communicates over the loopback adapter.
#preflight none

[CL 24751675 by Ben Marsh in ue5-main branch]
2023-03-22 14:34:19 -04:00
Ben Marsh
e96e9a14f8 EpicGames.Core: Allow adding arbitrary processes to ManagedProcessGroup objects (eg. processes created with regular C# Process constructs).
#preflight none

[CL 24751669 by Ben Marsh in ue5-main branch]
2023-03-22 14:34:06 -04:00
Helge Mathee
59aec7a444 RigVM: Support for uint32 types
#rb sara.schvartzman
#fyi jaime.cifuentes
#jira UE-180882
#preflight https://horde.devtools.epicgames.com/job/641ad78bec01de16641c96f4

[CL 24745737 by Helge Mathee in ue5-main branch]
2023-03-22 07:52:30 -04:00