50 Commits

Author SHA1 Message Date
ben marsh
8b875555d4 Horde: Merging latest from Main.
#rnx

[CL 32555579 by ben marsh in 5.4 branch]
2024-03-27 16:03:42 -04:00
ben marsh
7514b607a9 Horde: Merging latest from Main.
#rnx

[CL 32513392 by ben marsh in 5.4 branch]
2024-03-26 12:40:06 -04:00
ben marsh
377bb34a2a Horde: Merging latest from Main.
#rnx

[CL 32500076 by ben marsh in 5.4 branch]
2024-03-26 02:50:43 -04:00
ben marsh
35ae0d864a Horde: Merging latest from Main.
#rnx

[CL 32499238 by ben marsh in 5.4 branch]
2024-03-26 02:16:39 -04:00
ben marsh
fc03e6b7ff Horde: Merging latest from Main.
#rnx

[CL 32103260 by ben marsh in 5.4 branch]
2024-03-07 17:26:39 -05:00
ben marsh
973ec6dc3f Horde: Merging latest from Main.
[CL 31511277 by ben marsh in 5.4 branch]
2024-02-14 23:40:59 -05:00
ben marsh
bde862d1dc Horde: Merging latest from Main.
[CL 31423154 by ben marsh in 5.4 branch]
2024-02-13 09:03:57 -05:00
ben marsh
44b1046f15 Horde: Merge latest code from Main.
[CL 31159334 by ben marsh in 5.4 branch]
2024-02-03 11:45:07 -05:00
ben marsh
fc10542181 Horde: Include hashes with refs, allowing IStorageClient to return IBlobRef rather than IBlobHandle instances. Also rename methods that return blobs pointed to by refs as 'GetRefTarget'.
[CL 30960642 by ben marsh in 5.4 branch]
2024-01-28 16:27:16 -05:00
ben marsh
0776e0c167 Horde: Stop making object stores and storage backends disposable by default.
Lifetime management is difficult (and very messy) for these objects because blob handles can outlive configured backend/store instances. Rather than scoping lifetimes to each config file update, lifetimes are now managed by factory instances, which allow sharing resources between backends and only releasing them when the application stops.

[CL 30846935 by ben marsh in ue5-main branch]
2024-01-24 11:25:25 -05:00
ben marsh
43b7878c3c Horde: Include the Content-Length header when returning single file downloads, so browser can display progress bar.
[CL 30814380 by ben marsh in ue5-main branch]
2024-01-23 14:53:18 -05:00
ben marsh
a4265b060a Horde: Refactor of IStorageClient/IStorageBackend.
* IStorageBackend now includes methods for manipulating aliases and refs. All interaction with blobs is done through BlobLocators. This is the level of abstraction that the HTTP backend / controllers operate at.
* IStorageClient only manipulates blobs via handles, which abstracts any packing/compression/buffering that BundleStorageClient does.
* IBlobHandles no longer needs to include methods for partial reads, nesting, etc... Nesting of blobs within packets/bundles is now an implementation detail within the bundle storage client.

[CL 30648868 by ben marsh in ue5-main branch]
2024-01-16 18:08:13 -05:00
ben marsh
9ccdc40f9c Horde: Allow customizing blob serializers to use in different contexts, allowing us to lock serialization formats when talking to remote agents of a particular version.
To support this, the Node base class and serialization methods have been removed, and serialization is performed through a BlobConverter<T> instance created and cached according to a BlobSerializationOptions object. Custom instances of serializers with different configurations may be registered with this object, allowing different code paths to use different serialization settings.

#jira UE-203605

[CL 30591420 by ben marsh in ue5-main branch]
2024-01-12 08:59:27 -05:00
ben marsh
469964bbe4 Horde: Add commands for downloading/uploading tools from command line tool.
#jira

[CL 30211945 by ben marsh in ue5-main branch]
2023-12-08 11:34:16 -05:00
ben marsh
e75817e3fa Horde: Add a flag for whether to show tools inside UGS.
#jira

[CL 30206522 by ben marsh in ue5-main branch]
2023-12-07 23:29:46 -05:00
ben marsh
da16dc120c Horde: Include the current deployment identifier in tool responses.
#jira

[CL 30206253 by ben marsh in ue5-main branch]
2023-12-07 23:00:15 -05:00
ben marsh
e2d2d8ecdd Horde: Fix comment on tools controller.
#jira

[CL 30183566 by ben marsh in ue5-main branch]
2023-12-07 12:28:18 -05:00
ben marsh
5c55338f1b Horde: Move API classes for tools endpoint into EpicGames.Horde.
[CL 30116633 by ben marsh in ue5-main branch]
2023-12-05 08:46:52 -05:00
ben marsh
4021414faf Horde: Do not dispose of storage clients until all data has been read from streams that use them.
- Adds a new StreamWrapper class to EpicGames.Core which wraps another stream and forwards all overridable methods to it.
- Adds an extension method to create a StreamWrapper that overrides Dispose/DisposeAsync methods to dispose of other owned resources.

[CL 29935350 by ben marsh in ue5-main branch]
2023-11-27 11:14:39 -05:00
ben marsh
66829761be Horde: Change BlobHandle to be an interface so writer interfaces can also implement it.
#jira

[CL 29509421 by ben marsh in ue5-main branch]
2023-11-06 20:47:29 -05:00
ben marsh
edcd1c746c Horde: Remove IServerStorageClient, and fix race condition evaluating auth for storage client access.
* Both client and server implementations can now use the vanilla IStorageClient implementation, which now includes methods for requesting HTTP redirects.
* On the server, IStorageClient instances can be created by injecting a IStorageClientFactory instance, scoped to the request lifetime. This ensures the same configuration passed in via IOptionsSnapshot<> will be used, and any auth requests will be valid against the same configuration.

#jira

[CL 28789141 by ben marsh in ue5-main branch]
2023-10-14 18:58:40 -04:00
ben marsh
a187ace928 Horde: Allow bundles to be treated as regular blobs in the storage service. If a blob locator has a fragment specifier, it refers to a node, otherwise it refers to a bundle. The same operations for a bundle are available on BlobHandle.
#jira

[CL 28775324 by ben marsh in ue5-main branch]
2023-10-13 17:54:26 -04:00
ben marsh
6468512a55 Horde: Replace BundleLocator with BlobLocator.
#jira

[CL 28767004 by ben marsh in ue5-main branch]
2023-10-13 15:35:00 -04:00
ben marsh
631b1a42c3 Horde: Resurrect the BlobLocator type for uniquely identifying a blob via a string, allowing it to be sent over the wire and converted back into a BlobHandle via IStorageClient.
#jira

[CL 28591284 by ben marsh in ue5-main branch]
2023-10-09 14:22:54 -04:00
Ben Marsh
b628fc0627 Horde: Remove hashes from the core blob/node classes. Hashes are typically introduced at the application level (ie. in the serialized data) and don't need to be duplicated in the bundle headers.
[CL 28573294 by Ben Marsh in ue5-main branch]
2023-10-08 16:35:29 -04:00