Commit Graph

1938 Commits

Author SHA1 Message Date
ben marsh
93608838b3 Horde: Write leaf nodes, interior nodes, and directory nodes to separate streams for replicated Perforce data.
[CL 30785015 by ben marsh in ue5-main branch]
2024-01-22 16:07:30 -05:00
josh engebretson
31130cf904 Horde: Address naming consistency
#rnx

[CL 30784189 by josh engebretson in ue5-main branch]
2024-01-22 15:36:16 -05:00
ben marsh
bc47f9b56d Horde: Fix configuration of HTTP client used for interactive auth requests.
[CL 30783496 by ben marsh in ue5-main branch]
2024-01-22 15:01:56 -05:00
josh engebretson
0e0e8c23e3 Horde: Splitting Dashboard config out to top level, adding telemetry config and responses
#rnx

[CL 30783124 by josh engebretson in ue5-main branch]
2024-01-22 14:46:06 -05:00
ben marsh
ffb5dbdace Horde: Disable validation for object keys passed to the object store from the legacy V1 endpoint.
[CL 30774395 by ben marsh in ue5-main branch]
2024-01-22 10:53:10 -05:00
ben marsh
b6e2a435b4 Horde: Allow uppercase letters in object keys.
[CL 30771626 by ben marsh in ue5-main branch]
2024-01-22 09:44:47 -05:00
mikko mononen
d6c828e080 StateTree: Introduced TStateTreePropertyRef - type-safe FStateTreePropertyRef wrapper
#rb mikko.mononen tim.smith

[CL 30764929 by mikko mononen in ue5-main branch]
2024-01-22 05:01:31 -05:00
ben marsh
7dac926f00 Horde: Query the server API version before uploading tool data, and fall back to using older storage formats if the server requires it.
[CL 30762520 by ben marsh in ue5-main branch]
2024-01-21 13:52:05 -05:00
ben marsh
d4b5542320 Horde: Sanitize object keys before writing artifacts to storage.
[CL 30762397 by ben marsh in ue5-main branch]
2024-01-21 13:04:05 -05:00
ben marsh
fbdf7c2c08 Horde: Add an API version number that can be queried from the /api/v1/server/info endpoint. Allows coordinating client requests to the appropriate server version.
[CL 30762039 by ben marsh in ue5-main branch]
2024-01-21 11:40:27 -05:00
ben marsh
68fb925251 Horde: Add a singleton IHordeClient object which allows constructing new HordeHttpClient instances. Allows keeping all the configuration/state in one place.
[CL 30761845 by ben marsh in ue5-main branch]
2024-01-21 10:33:11 -05:00
ben marsh
b9dffa8c19 Horde: Add a configuration provider for reading registry settings on Windows, and remove convoluted logic to determine default ports to run server on.
Fixes inconsistencies between port numbers used by the server and port numbers returned to clients via the /api/v1/server/ports endpoint, which are derived from the ServerSettings object, causing connection failures.

[CL 30690641 by ben marsh in ue5-main branch]
2024-01-18 13:49:50 -05:00
ben marsh
4bc7aa8795 Horde: Fix support for extracting v2 format chunked blob data, and restore change to use fast memory mapped files for writing data on Windows.
[FYI] Carl.Bystrom

[CL 30688622 by ben marsh in ue5-main branch]
2024-01-18 12:43:57 -05:00
ben marsh
f7b403735e Horde: Temporarily revert change to use custom code path for extracting files from storage system on Windows. Suspect some bad interaction with WINE.
[CL 30683713 by ben marsh in ue5-main branch]
2024-01-18 10:14:44 -05:00
ben marsh
1afccc777a Horde: Verify the contents of files written through compute leases after writing them.
[CL 30683067 by ben marsh in ue5-main branch]
2024-01-18 09:39:32 -05:00
ben marsh
e1673244d9 Fix static analysis warning.
[CL 30675874 by ben marsh in ue5-main branch]
2024-01-17 19:21:16 -05:00
ben marsh
176b09a101 Horde: Add a constructor for MemoryMappedView which uses P/Invoke to map the file into memory. MemoryMappedViewAccessor calls Flush when disposed, which introduces a significant performance hit.
[CL 30674957 by ben marsh in ue5-main branch]
2024-01-17 18:53:47 -05:00
ben marsh
75872c1dd9 Horde: Allow all StringId characters within object keys, and fix invalid paths being used for prefixed object stores.
[CL 30669957 by ben marsh in ue5-main branch]
2024-01-17 16:56:59 -05:00
ben marsh
f0e4c496ae Horde: Merge IStorageWriter and IBlobWriter.
[CL 30667967 by ben marsh in ue5-main branch]
2024-01-17 15:55:37 -05:00
ben marsh
000eab0fff Horde: Improve performance of extracting data from bundles.
* Enumerating chunks to extract is now done asynchronously to extraction, and chunks are enumerated in order. This massively increases cache coherency over having separate writers run over different partitions of the data.
* Data is now written to disk using memory mapped files rather than buffered stream IO. Significantly faster in testing (~2x).

[CL 30660663 by ben marsh in ue5-main branch]
2024-01-17 11:55:39 -05:00
ben marsh
9399e51142 Horde: Fix some bundle V2 decode errors introduced in CL 30648855
[CL 30658028 by ben marsh in ue5-main branch]
2024-01-17 09:46:11 -05:00
carl bystrom
f9452171c3 Horde: Avoid sending cancellation exceptions during compute execution
Shows up in client's log even though it's benign.

[CL 30656339 by carl bystrom in ue5-main branch]
2024-01-17 07:37:32 -05:00
ben marsh
54408859da Horde: Allow periods in ref names, as long as they follow the same requirements as filesystem paths (not appearing at the start or end of a path fragment).
[CL 30649104 by ben marsh in ue5-main branch]
2024-01-16 18:20:23 -05:00
ben marsh
a4265b060a Horde: Refactor of IStorageClient/IStorageBackend.
* IStorageBackend now includes methods for manipulating aliases and refs. All interaction with blobs is done through BlobLocators. This is the level of abstraction that the HTTP backend / controllers operate at.
* IStorageClient only manipulates blobs via handles, which abstracts any packing/compression/buffering that BundleStorageClient does.
* IBlobHandles no longer needs to include methods for partial reads, nesting, etc... Nesting of blobs within packets/bundles is now an implementation detail within the bundle storage client.

[CL 30648868 by ben marsh in ue5-main branch]
2024-01-16 18:08:13 -05:00
ben marsh
117c011d1d EpicGames.Core: Fix incorrect data returned by ReadOnlyMemoryArray enumerator.
[CL 30643074 by ben marsh in ue5-main branch]
2024-01-16 15:32:50 -05:00