Commit Graph

39 Commits

Author SHA1 Message Date
reapazor
b8f790f8e6 PR #11757: Fix tool deployment from Horde Agents. Specifying a JSON options file is now optional; if empty, the settings will be read from the environment.
#jira UE-212503

[CL 33038950 by reapazor in ue5-main branch]
2024-04-17 12:22:42 -04:00
ben marsh
d95988bb31 BuildGraph: Log the tool id when deploying to Horde.
[CL 31159503 by ben marsh in ue5-main branch]
2024-02-03 12:59:37 -05:00
ben marsh
a7a7d64386 Build fix.
[CL 30960714 by ben marsh in ue5-main branch]
2024-01-28 16:36:08 -05:00
ben marsh
ab80ea0a41 Horde: Add separate types for blob handles with a hash (IBlobRef) and blob handles with a hash and serialization options (IBlobRef<T>). Storing serialization options inside IBlobRef<T> implementations removes a lot of boilerplate code to pass options objects around, and ensures consistency when navigating object hierarchies.
[CL 30957772 by ben marsh in ue5-main branch]
2024-01-27 19:24:46 -05:00
ben marsh
7dac926f00 Horde: Query the server API version before uploading tool data, and fall back to using older storage formats if the server requires it.
[CL 30762520 by ben marsh in ue5-main branch]
2024-01-21 13:52:05 -05:00
ben marsh
f0e4c496ae Horde: Merge IStorageWriter and IBlobWriter.
[CL 30667967 by ben marsh in ue5-main branch]
2024-01-17 15:55:37 -05:00
ben marsh
a4265b060a Horde: Refactor of IStorageClient/IStorageBackend.
* IStorageBackend now includes methods for manipulating aliases and refs. All interaction with blobs is done through BlobLocators. This is the level of abstraction that the HTTP backend / controllers operate at.
* IStorageClient only manipulates blobs via handles, which abstracts any packing/compression/buffering that BundleStorageClient does.
* IBlobHandles no longer needs to include methods for partial reads, nesting, etc... Nesting of blobs within packets/bundles is now an implementation detail within the bundle storage client.

[CL 30648868 by ben marsh in ue5-main branch]
2024-01-16 18:08:13 -05:00
ben marsh
9ccdc40f9c Horde: Allow customizing blob serializers to use in different contexts, allowing us to lock serialization formats when talking to remote agents of a particular version.
To support this, the Node base class and serialization methods have been removed, and serialization is performed through a BlobConverter<T> instance created and cached according to a BlobSerializationOptions object. Custom instances of serializers with different configurations may be registered with this object, allowing different code paths to use different serialization settings.

#jira UE-203605

[CL 30591420 by ben marsh in ue5-main branch]
2024-01-12 08:59:27 -05:00
ben marsh
4373db4c84 Horde: Add helper methods for writing directory trees to storage writers.
[CL 30292017 by ben marsh in ue5-main branch]
2023-12-13 09:45:27 -05:00
ryan hummer
583435d379 [Backout] - CL30186989
[FYI] joshua.shlemmer
Original CL Desc
-----------------------------------------------------------------

Modifies buildtasks that were using HTTPClients so that they use a singleton http client.

Follow up from CL 30122407

#rb Eric.Knapik, Wes.Hunt

[CL 30210278 by ryan hummer in ue5-main branch]
2023-12-08 09:42:30 -05:00
joshua shlemmer
e4a921891d Modifies buildtasks that were using HTTPClients so that they use a singleton http client.
Follow up from CL 30122407

#rb Eric.Knapik, Wes.Hunt

[CL 30187247 by joshua shlemmer in ue5-main branch]
2023-12-07 13:57:33 -05:00
ben marsh
66829761be Horde: Change BlobHandle to be an interface so writer interfaces can also implement it.
#jira

[CL 29509421 by ben marsh in ue5-main branch]
2023-11-06 20:47:29 -05:00
ben marsh
ccc5dfdf8a Horde: Rename BundleReaderCache to BundleCache. Will be used for tracking write data also.
#jira

[CL 29495211 by ben marsh in ue5-main branch]
2023-11-06 14:14:33 -05:00
ben marsh
24bda4ce70 Horde: Remove BundleNodeHandle and BundleNodeLocator in favor of storing parsed data on handle objects.
#jira

[CL 29005756 by ben marsh in ue5-main branch]
2023-10-21 22:46:54 -04:00
ben marsh
00ef101197 Horde: Unify BundleStorageClient and BundleStorageClientWrapper.
#jira

[CL 29001725 by ben marsh in ue5-main branch]
2023-10-21 15:08:34 -04:00
ben marsh
9a598cd6cb Horde: Make HttpStorageClient a direct implementation of IStorageClient rather than deriving from BundleStorageClient.
#jira

[CL 29001200 by ben marsh in ue5-main branch]
2023-10-21 14:38:40 -04:00
Ben Marsh
3b7d176fdd Horde: Use a cache on the local disk to wrap any requests for bundle data.
[CL 28132629 by Ben Marsh in ue5-main branch]
2023-09-22 09:22:32 -04:00
Ben Marsh
ae3ad9b5c0 Fix build error.
[CL 28114134 by Ben Marsh in ue5-main branch]
2023-09-21 19:22:29 -04:00
Ben Marsh
feaf55847d Horde: Fix UAT build error.
[CL 28114013 by Ben Marsh in ue5-main branch]
2023-09-21 19:20:20 -04:00
Ben Marsh
61337286c7 Build fix.
[CL 28056869 by Ben Marsh in ue5-main branch]
2023-09-20 17:07:37 -04:00
Ben Marsh
0e4f348199 Horde: Move IStorageClient implementations into the EpicGames.Horde.Storage.Clients namespace.
[CL 27738432 by Ben Marsh in ue5-main branch]
2023-09-09 10:14:12 -04:00
Ben Marsh
ba352e05af Horde: Add a custom StorageCache implementation rather than using IMemoryCache.
This creates separate arenas for different types of content, improving read performance due to header data being discarded, and also allows cache instances to be globally configured and injected via DI.

[CL 27528676 by Ben Marsh in ue5-main branch]
2023-08-31 13:35:03 -04:00
Ben Marsh
31a83fc628 Horde: Move bundle implementation into a separate namespace.
[CL 26226699 by Ben Marsh in ue5-main branch]
2023-06-24 16:37:55 -04:00
Ben Marsh
b7f9afb66a Horde: Add a custom handle type for nodes stored in bundles, abstracting implementation details away from most clients.
[CL 26226673 by Ben Marsh in ue5-main branch]
2023-06-24 16:22:48 -04:00
Ben Marsh
4819131d52 Build fix.
[CL 25904402 by Ben Marsh in ue5-main branch]
2023-06-09 16:57:16 -04:00