Commit Graph

1611 Commits

Author SHA1 Message Date
Ben Marsh
5f5fd406f1 Horde: Separate implementation of the local cache from the backend implementation, allowing multiple backends to share the same cache.
[CL 27741582 by Ben Marsh in ue5-main branch]
2023-09-10 12:34:12 -04:00
Ben Marsh
3fac84941c Horde: Add a caching storage backend which uses the local filesystem.
[CL 27741436 by Ben Marsh in ue5-main branch]
2023-09-10 11:03:30 -04:00
Ben Marsh
ff075c94a0 Horde: Implement BundleStorageClient using a user-supplied IStorageBackend implementation. This allows constructing a hierarchy of backends separately to metadata operations, allowing us to wrap backends in caching layers.
[CL 27740032 by Ben Marsh in ue5-main branch]
2023-09-09 17:54:42 -04:00
Ben Marsh
3d14c169a9 Fix null reference warning.
[CL 27739364 by Ben Marsh in ue5-main branch]
2023-09-09 15:08:33 -04:00
Ben Marsh
00ae754c4f Horde: Add an HTTP storage backend, which uses the regular Horde endpoints for reading and writing blobs.
[CL 27739157 by Ben Marsh in ue5-main branch]
2023-09-09 14:08:45 -04:00
Ben Marsh
dd78ef8dd7 Horde: Deprecate writing an explicit path through IStorageBackend; add an override which allows the backend to generate a unique id. This unifies server-side codepaths (where the generation of a unique id is trusted) and client-side codepaths (where it is not). By forcing unique ids to be generated on the server, we can further abstract logic around creating bundles and make it more of an optimization over storing small blobs.
[CL 27739032 by Ben Marsh in ue5-main branch]
2023-09-09 13:20:37 -04:00
Ben Marsh
a6c4a6bc23 Missing file.
[CL 27738538 by Ben Marsh in ue5-main branch]
2023-09-09 10:36:45 -04:00
Ben Marsh
3da367ffae Horde: Move storage backend implementations into EpicGames.Horde.
[CL 27738534 by Ben Marsh in ue5-main branch]
2023-09-09 10:36:24 -04:00
Ben Marsh
0e4f348199 Horde: Move IStorageClient implementations into the EpicGames.Horde.Storage.Clients namespace.
[CL 27738432 by Ben Marsh in ue5-main branch]
2023-09-09 10:14:12 -04:00
andrew phillips
bf12dcee39 Fixing vulnerable NuGet deps.
System.Drawing.Common 4.7.0 => 4.7.2  (https://github.com/advisories/GHSA-rxg9-xrhp-64gj)
MongoDB.Driver 2.11.5 => 2.19.0  (https://github.com/advisories/GHSA-7j9m-j397-g4wx)

#jira UE-194453
#rb francis.hurteau
#lockdown michal.valient

[CL 27715945 by andrew phillips in ue5-main branch]
2023-09-08 12:44:45 -04:00
Ben Marsh
406c88ea66 Horde: Increase max number of concurrent tasks to 16.
[CL 27713265 by Ben Marsh in ue5-main branch]
2023-09-08 11:45:52 -04:00
Ben Marsh
46884ce24d Horde: Include download speed for log events output when retrieving bundles.
[CL 27710844 by Ben Marsh in ue5-main branch]
2023-09-08 10:26:51 -04:00
Ben Marsh
c4821bd030 Horde: Add linear hash for files to FileEntry instances, rather than using the hash of the Merkle tree underneath it. Allows verifying integrity of files extracted to the workspace.
[CL 27709404 by Ben Marsh in ue5-main branch]
2023-09-08 09:22:49 -04:00
Ben Marsh
b2a17d9aa1 Horde: Fix null reference warning.
[CL 27708760 by Ben Marsh in ue5-main branch]
2023-09-08 08:46:22 -04:00
Ben Marsh
f2b085674a Horde: Output stats when copying data from temp storage.
[CL 27708691 by Ben Marsh in ue5-main branch]
2023-09-08 08:39:49 -04:00
Ben Marsh
859a372474 Horde: Fix warnings highlighted by VS 17.8 preview static analyzer.
[CL 27708585 by Ben Marsh in ue5-main branch]
2023-09-08 08:33:18 -04:00
Ben Marsh
eb6bc84881 Horde: Fix items moved from workspace cache not being correctly added back into the hash lookup.
[CL 27699650 by Ben Marsh in ue5-main branch]
2023-09-07 21:20:50 -04:00
Ben Marsh
ef916d846b Horde: Prevent too many exports being serialized into a single bundle section.
[CL 27699152 by Ben Marsh in ue5-main branch]
2023-09-07 21:04:05 -04:00
Ben Marsh
154c57115b Horde: Missing comment.
#jira UE-194779

[CL 27696859 by Ben Marsh in ue5-main branch]
2023-09-07 20:06:02 -04:00
Ben Marsh
ff0658eb36 Missing quote.
[CL 27696541 by Ben Marsh in ue5-main branch]
2023-09-07 19:58:51 -04:00
Ben Marsh
17e04db780 Horde: Add checks to ensure values written to bundles are not out of range.
[CL 27696473 by Ben Marsh in ue5-main branch]
2023-09-07 19:57:21 -04:00
Ben Marsh
694903863b Horde: Better logging for temp storage stats.
[CL 27695773 by Ben Marsh in ue5-main branch]
2023-09-07 19:34:37 -04:00
Ben Marsh
0dac7d9988 Horde: Add stat logging for temp storage retrieval using bundles.
[CL 27692662 by Ben Marsh in ue5-main branch]
2023-09-07 18:24:35 -04:00
Ben Marsh
3ab9e2eb5d Horde: Move unused files to the cache during a workspace sync, allowing them to be reused for replacement files.
[CL 27679980 by Ben Marsh in ue5-main branch]
2023-09-07 14:07:04 -04:00
carl bystrom
47fae020bb Horde: Make parameters to ManagedWorkspace configurable
[CL 27645033 by carl bystrom in ue5-main branch]
2023-09-06 14:31:48 -04:00